From c4d2a8f507c611725f90f464e27322d9d1dabfe8 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:11:20 -0700 Subject: [PATCH 01/73] repo-qa-local: use lab.mcp, let makeSession build the MCP tools Synced from locallmlab-sdk. The example made its own MCPServerManager next to LocalLMLab's. It now adds Deepwiki to lab.mcp, enables the tools it uses (a new server's tools start disabled; read_wiki_contents stays off) and calls lab.makeSession(route:instructions:includeMCPTools: true). Same tools offered, now tagged as MCP for policy and records. README: the diff table against repo-qa, and the verified-live transcript (Deepwiki's tool is now ask_wiki_question). Package.swift: header comment only. Builds against 1.0.0-RC.1; verified live with Qwen3-8B (facebook/react). Co-Authored-By: Claude --- examples/repo-qa-local/Package.swift | 2 +- examples/repo-qa-local/README.md | 33 +++++++++------ .../Sources/RepoQALocal/main.swift | 42 ++++++++++--------- 3 files changed, 44 insertions(+), 33 deletions(-) diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index 5012def..ca579c5 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -2,7 +2,7 @@ import Foundation import PackageDescription -// repo-qa-local — repo-qa's exact MCPTool setup, but the answer comes from a locally-run +// repo-qa-local — repo-qa's Deepwiki setup (server in lab.mcp), but the answer comes from a locally-run // open-weight (MLX) model routed through the 1.0 model layer instead of Apple's on-device model. // Like code-buddy, it links BOTH SDK binaries: LocalLMLabSDKCore.xcframework AND // LocalLMLabSDKInference.xcframework (the MLX runtime — mlx-swift-lm + Metal statically linked). diff --git a/examples/repo-qa-local/README.md b/examples/repo-qa-local/README.md index 75c7a88..f52aff6 100644 --- a/examples/repo-qa-local/README.md +++ b/examples/repo-qa-local/README.md @@ -13,8 +13,9 @@ once on first run. The part of the SDK that makes this manageable is the **model layer**: you name a model, and it checks the model fits in this Mac's memory, downloads it with a progress bar, loads and unloads it as needed, and hands you a chat session — the *same* session API you'd use for Apple's model -or Claude. Moving from `repo-qa` to this version is about 20 lines of change; the Deepwiki / MCP -half of the code is byte-for-byte identical. +or Claude. Moving from `repo-qa` to this version is a small change; the Deepwiki / MCP half +keeps repo-qa's shape, with the server added to the lab's manager (`lab.mcp`) instead of one of +its own. **What it highlights for SDK developers:** @@ -22,14 +23,17 @@ half of the code is byte-for-byte identical. it, point a name at it, open a session. [`code-buddy`](../code-buddy) is the full version (two models, routing between them, streamed output, an agent loop). - **The model layer is a swap-in, not a rewrite.** Diff this `main.swift` against `repo-qa`'s: - the tool-building code is unchanged; only how the session is created differs - ([table below](#what-the-model-layer-adds-diff-against-repo-qa)). + the same server, the same tools; what changes is where the MCP manager lives (`lab.mcp`) and how + the session is created ([table below](#what-the-model-layer-adds-diff-against-repo-qa)). +- **One MCP manager: `lab.mcp`.** An app that uses `LocalLMLab` adds its servers to `lab.mcp` and + lets `makeSession` build the MCP tools; it never makes an `MCPServerManager` of its own. (repo-qa + has no `LocalLMLab`, so its own manager is right there.) - **Any MLX model, or fall back to Apple's.** `--model ` runs a different open-weight model; `--apple` uses Apple's on-device model instead — the same code path handles all of them. > **Want the on-device-model version instead?** [`repo-qa`](../repo-qa) is this exact tool with > `SystemLanguageModel.default` and no model layer. Diff the two `main.swift`s to see what the -> model layer adds — it's about 20 lines. +> model layer adds. ## Quick start @@ -152,16 +156,18 @@ new version, then update the repo and its commit together. See ## What the model layer adds (diff against `repo-qa`) -The Deepwiki / `MCPTool` half of `main.swift` is a verbatim copy of `repo-qa` — connecting, -building an `MCPTool` per tool from its live schema, excluding `read_wiki_contents` (same -reasons; see `repo-qa`'s README). The only differences: +The Deepwiki half of `main.swift` follows `repo-qa` — connecting, offering each tool from its +live schema, excluding `read_wiki_contents` (same reasons; see `repo-qa`'s README). The +differences: | `repo-qa` | `repo-qa-local` | |---|---| | `import LocalLMLabSDKCore` | `+ import LocalLMLabSDKInference` | | `SystemLanguageModel.default` availability check | `MLXModelProvider` + `LocalLMLab` + `lab.models.route(.local, to: …)` | | — | `mlx.validate` → `mlx.download` (streamed) if the weights aren't local yet | -| `LanguageModelSession(tools: tools) { instructions }` | `lab.makeSession(route: .local, tools: tools, instructions:)` | +| `let manager = MCPServerManager()` → `manager.addServer(…)` | `lab.mcp.addServer(…)` — the lab's manager; never a second one | +| an `MCPTool(descriptor:manager:)` per tool | `lab.mcp.setToolEnabled(server:tool:enabled: true)` per tool — a new server's tools start **disabled** (security default), so without this the model gets no tools | +| `LanguageModelSession(tools: tools) { instructions }` | `lab.makeSession(route: .local, instructions:, includeMCPTools: true)` — builds the `MCPTool`s from `lab.mcp`'s enabled tools | | `session.respond(to:)` | `session.languageModelSession.respond(to:)` — same FoundationModels session underneath | That's the point: the model layer is a swap-in, not a rewrite. @@ -175,20 +181,21 @@ That's the point: the model layer is a swap-in, not a rewrite. ## Verified live ``` -model: mlx:mlx-community/Qwen3-8B-4bit · SDK 1.0.0-RC.1 +model: mlx:mlx-community/Qwen3-8B-4bit · SDK 1.0.0-GA +pinned to 545dc42 (shipped with this example) Connecting to Deepwiki… Skipping read_wiki_contents: excluded by this example. -Built 2 tool(s) from Deepwiki's live schema: ask_question, read_wiki_structure +Enabled 2 tool(s) from Deepwiki's live schema: ask_wiki_question, read_wiki_structure Asking: Regarding the GitHub repository "facebook/react": What does this repo do? -… the response provided a detailed breakdown of the React monorepo structure … +… The tool's response provided a detailed breakdown of the repository's components and purpose … The **facebook/react** repository is the **monorepo for React**, a JavaScript library for building user interfaces. [...] ``` -A real, unmocked run — Qwen3-8B-4bit called Deepwiki's `ask_question` and answered from the +A real, unmocked run — Qwen3-8B-4bit called Deepwiki's `ask_wiki_question` and answered from the repo's actual docs. (The `…` block is the model's own reasoning; the 1.0-beta MLX bridge streams it through inline — carve it out consumer-side if you want a clean answer.) The `--apple` path was confirmed the same way against the on-device model. diff --git a/examples/repo-qa-local/Sources/RepoQALocal/main.swift b/examples/repo-qa-local/Sources/RepoQALocal/main.swift index 2a16feb..2a3e18f 100644 --- a/examples/repo-qa-local/Sources/RepoQALocal/main.swift +++ b/examples/repo-qa-local/Sources/RepoQALocal/main.swift @@ -1,7 +1,8 @@ // repo-qa-local — repo-qa, but the answer comes from an open-weight model you download and run -// locally (via MLX) instead of Apple's on-device model. The Deepwiki / MCPTool half is a -// verbatim copy of repo-qa's; the only difference is the ~20 lines that set up the 1.0 model -// layer and swap `LanguageModelSession(tools:)` for `lab.makeSession(route:tools:)`. +// locally (via MLX) instead of Apple's on-device model. The Deepwiki half follows repo-qa's, with +// one change: the MCP server goes into `lab.mcp` (the lab's manager — an app that uses LocalLMLab +// never makes an `MCPServerManager` of its own), and `lab.makeSession(route:)` builds the MCP tools +// from it, instead of repo-qa's own manager + `MCPTool`s + `LanguageModelSession(tools:)`. // // swift run RepoQALocal anthropics/claude-code "What is the plugin system?" // swift run RepoQALocal facebook/react # default question @@ -77,11 +78,10 @@ func run() async { note("pinned to \(pin.revision.prefix(7)) (\(pin.source == .shipped ? "shipped with this example" : "first download"))") } - // --- everything below is repo-qa, unchanged --- + // --- the Deepwiki half: repo-qa's, with the server in lab.mcp --- - let manager = MCPServerManager() note("Connecting to Deepwiki…") - let connectResult = await manager.addServer( + let connectResult = await lab.mcp.addServer( url: URL(string: "https://mcp.deepwiki.com/mcp")!, displayName: "Deepwiki" ) @@ -89,29 +89,33 @@ func run() async { note("Could not connect to Deepwiki: \(connectResult)"); return } - // Build a Tool for each of Deepwiki's tools from its own live schema. `read_wiki_contents` is - // skipped by name: it dumps a repo's entire wiki unscoped (~165K tokens for anthropics/ - // claude-code in one call), which `MCPTool` can't know from the schema — that curation is the - // app's job (see docs/sdk-guide.md §3). A tool whose schema doesn't build is skipped, not fatal. - var tools: [any Tool] = [] + // A new server's tools start disabled (a security default: nothing reaches the model until the + // host opts in), so enable the ones this example uses. `read_wiki_contents` stays off: it dumps + // a repo's entire wiki unscoped (~165K tokens for anthropics/claude-code in one call), which + // `MCPTool` can't know from the schema — that curation is the app's job (see + // docs/sdk-guide.md §3). A tool whose schema doesn't build is skipped, not fatal (makeSession + // would otherwise fail on it). + var enabled: [String] = [] for descriptor in state.tools { guard descriptor.name != "read_wiki_contents" else { note("Skipping \(descriptor.name): excluded by this example.") continue } - do { tools.append(try MCPTool(descriptor: descriptor, manager: manager)) } - catch { note("Skipping \(descriptor.name): \(error)") } + do { _ = try MCPTool(descriptor: descriptor, manager: lab.mcp) } + catch { note("Skipping \(descriptor.name): \(error)"); continue } + lab.mcp.setToolEnabled(server: state.id, tool: descriptor.name, enabled: true) + enabled.append(descriptor.name) } - guard !tools.isEmpty else { note("Deepwiki didn't offer any usable tools."); return } - note("Built \(tools.count) tool(s) from Deepwiki's live schema: \(tools.map(\.name).joined(separator: ", "))") + guard !enabled.isEmpty else { note("Deepwiki didn't offer any usable tools."); return } + note("Enabled \(enabled.count) tool(s) from Deepwiki's live schema: \(enabled.joined(separator: ", "))") - // The one line that changes from repo-qa: lab.makeSession(route:tools:) instead of - // LanguageModelSession(tools:) — same FoundationModels session underneath, just backed by - // whichever model the route points at. + // Where repo-qa makes `LanguageModelSession(tools:)`, this calls lab.makeSession(route:) — same + // FoundationModels session underneath, backed by whichever model the route points at, with + // lab.mcp's enabled tools included (`includeMCPTools`, on by default). let instructions = "You answer questions about GitHub repositories using the documentation tools available to you. Always ground your answer in what the tools actually return — don't answer from general knowledge if a tool call would give a more specific, current answer." let session: LocalLMLabSession do { - session = try lab.makeSession(route: .local, tools: tools, instructions: instructions, includeMCPTools: false) + session = try lab.makeSession(route: .local, instructions: instructions, includeMCPTools: true) } catch { note("makeSession failed: \(error)"); exit(1) } From 2a418d801731cba09156ad626d9e1a152a648585 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:35:18 -0700 Subject: [PATCH 02/73] aiql: data tools from lab.mcp, file-backed; build against 2.0.0-dev Synced from locallmlab-sdk. AIQL made its own MCPServerManager and wrapped the server's tools with FileBackedTool.mcp. It now adds the server to lab.mcp, enables its top 4 data tools, sets lab.mcp.setFileBackedOutput(MCPFileBackedOutput(root:...), server:) for the run's folder, and lets makeSession(includeMCPTools: true) offer them with saveAs. Servers from an earlier Go are disconnected so their tools don't reach the model. setFileBackedOutput is new in 2.0, so Package.swift defaults to 2.0.0-dev (new knownSDKReleases entry; checksums from the release's .sha256 files). Kept 1.0.0-RC.1 as the previous release, dropped 1.0.0-beta.4. README mirrored. swift build and xcodebuild (AIQL.xcodeproj) succeed against v2.0.0-dev. Not run end to end. Co-Authored-By: Claude --- examples/aiql/Package.swift | 21 ++++----- examples/aiql/README.md | 12 ++--- examples/aiql/Sources/AIQL/AIQLApp.swift | 56 +++++++++++++++--------- 3 files changed, 52 insertions(+), 37 deletions(-) diff --git a/examples/aiql/Package.swift b/examples/aiql/Package.swift index 7972c08..e771140 100644 --- a/examples/aiql/Package.swift +++ b/examples/aiql/Package.swift @@ -8,11 +8,12 @@ import PackageDescription // that writes a CSV into a folder you chose — pull the dataset (FileBackedTool, so the raw // payload never enters the model's context), then project / filter / sort with the Core data // verbs (docs/sdk-guide.md §8b). The row data never passes through the model, so it can't be -// fabricated. +// fabricated. The data tools come from lab.mcp, file-backed (`setFileBackedOutput`, new in +// 2.0.0) — so this example needs SDK 2.0.0-dev or later. // // Combines three existing examples: workspace-buddy-local (SwiftUI + App Sandbox + MLX model + -// folder picker), plate-today (MCP client + OAuth redirect wiring), repo-qa (MCPTool from a live -// schema). Like code-buddy / workspace-buddy-local it links BOTH SDK binaries: +// folder picker), plate-today (MCP client + OAuth redirect wiring), repo-qa (tools from a live +// MCP schema). Like code-buddy / workspace-buddy-local it links BOTH SDK binaries: // LocalLMLabSDKCore.xcframework AND LocalLMLabSDKInference.xcframework (the MLX runtime). // // Requires macOS 27 + Xcode 27 (the model layer is built on FoundationModels' `LanguageModel` @@ -30,21 +31,21 @@ struct SDKRelease { // uses. `knownSDKReleases` carries this plus the previous release. Build against another // published version: set LOCALLM_SDK_VERSION in your shell (works for `swift build` / CI, NOT // inside Xcode), or edit `defaultSDKVersion` here. -let defaultSDKVersion = "1.0.0-RC.1" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-beta.4": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-beta.4/LocalLMLabSDKCore-1.0.0-beta.4.xcframework.zip", - coreChecksum: "3ed0e79b6914e6b48b7ae27f3fdda139f71e3d60f603daf54901716c8c972cb3", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-beta.4/LocalLMLabSDKInference-1.0.0-beta.4.xcframework.zip", - inferenceChecksum: "fa8feb19883f9a465a69f39d756f1b41b515c8298c891b06fef5da5b81b2a03c" - ), "1.0.0-RC.1": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", coreChecksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKInference-1.0.0-RC.1.xcframework.zip", inferenceChecksum: "e24cb0581807d37a7b595f0b198b9a9eeecc1c5c36fb59f61429b8a7b42dc169" ), + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/aiql/README.md b/examples/aiql/README.md index 0d2e2bb..dc1ce4b 100644 --- a/examples/aiql/README.md +++ b/examples/aiql/README.md @@ -47,7 +47,7 @@ evaluates a condition, or handles a value. `sqlite3_set_authorizer` allowlist (SELECT/READ/FUNCTION only — no writes, no `ATTACH`), one statement, a timeout, a row cap. Only a receipt (columns, row count, first rows) returns - which tools the model even sees: the server's tools are ranked by how "dataset-like" the name - looks and only the top few are wrapped; the raw-file reader is withheld + looks and only the top few are enabled; the raw-file reader is withheld - the raw payload's path — `FileBackedTool`'s `saveAs` parks it in `raw/data.json`; it never enters the model's context @@ -74,8 +74,8 @@ Three existing examples stitched together, plus the SQL tools: | from | what it contributes | |---|---| | [`workspace-buddy-local`](../workspace-buddy-local) | SwiftUI + App Sandbox + `MLXModelProvider` download + `NSOpenPanel` folder picker + security-scoped bookmark | -| [`plate-today`](../plate-today) | `MCPServerManager` + the OAuth redirect wired through `AppDelegate` (not SwiftUI's `.onOpenURL`) + `CFBundleURLTypes` | -| [`repo-qa`](../repo-qa) | building tools from a live MCP schema — here `FileBackedTool.mcp(descriptor:manager:root:)` | +| [`plate-today`](../plate-today) | the MCP client + the OAuth redirect wired through `AppDelegate` (not SwiftUI's `.onOpenURL`) + `CFBundleURLTypes`. AIQL uses `LocalLMLab`, so its server goes into `lab.mcp`, the lab's manager — never an `MCPServerManager` of its own | +| [`repo-qa`](../repo-qa) | tools built from a live MCP schema — here by `makeSession` from `lab.mcp`: `setToolEnabled` for the chosen data tools (a new server's tools start disabled), and `lab.mcp.setFileBackedOutput(MCPFileBackedOutput(root:…), server:)` so each one is offered as a `FileBackedTool` with `saveAs` | | SDK §8b | `loadTable` + `sqlQuery` — JSON records → an ephemeral SQLite table → one read-only `SELECT` → CSV. `describeJson` / `csvInfo` for discovery/verification | Other things worth a look in `Sources/AIQL/AIQLApp.swift`: @@ -83,7 +83,7 @@ Other things worth a look in `Sources/AIQL/AIQLApp.swift`: - **`session.events`** — the pipeline's progress panel is driven by the session's tool-call side-channel (`SessionEvent.toolCallStarted` / `.toolCallFinished`), mapped to friendly labels. - **Tool curation for a small model** — the server's tools are ranked by how "dataset-like" the - name looks and only the top few are wrapped; `readWorkspaceFile` is deliberately not offered + name looks and only the top few are enabled; `readWorkspaceFile` is deliberately not offered (the raw dump is far past the context window, so the model only ever sees the bounded `describeJson` / `csvInfo` views). @@ -102,8 +102,8 @@ export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer ``` Skip it if `xcode-select -p` already points at Xcode 27 (adjust the path if yours lives elsewhere). Leaves your system default alone; lasts only for the current terminal (re-run it in each new one, -or add it to your `~/.zshrc`). `Package.swift` builds against SDK `1.0.0-RC.1` with no further -setup — it links **two** binaries, `LocalLMLabSDKCore.xcframework` and +or add it to your `~/.zshrc`). `Package.swift` builds against SDK `2.0.0-dev` with no further +setup (AIQL needs it: it uses `lab.mcp.setFileBackedOutput`, new in 2.0) — it links **two** binaries, `LocalLMLabSDKCore.xcframework` and `LocalLMLabSDKInference.xcframework` (the MLX runtime), from that one GitHub Release. `export LOCALLM_SDK_VERSION=` to pin a different published release. **No Metal Toolchain needed** — the prebuilt Inference xcframework bundles the compiled `default.metallib`; you only diff --git a/examples/aiql/Sources/AIQL/AIQLApp.swift b/examples/aiql/Sources/AIQL/AIQLApp.swift index 7521e4b..1fc9bfe 100644 --- a/examples/aiql/Sources/AIQL/AIQLApp.swift +++ b/examples/aiql/Sources/AIQL/AIQLApp.swift @@ -13,7 +13,7 @@ // // Structure borrowed from: workspace-buddy-local (SwiftUI + App Sandbox + MLX model + folder // picker + security-scoped bookmark), plate-today (MCP client + OAuth redirect via AppDelegate), -// repo-qa (MCPTool / FileBackedTool.mcp from a live schema). +// repo-qa (tools from a live MCP schema — here built by makeSession from lab.mcp, file-backed). import AppKit import Foundation @@ -95,7 +95,6 @@ final class AIQLModel: ObservableObject { @Published private(set) var stage: Stage = .idle @Published private(set) var steps: [String] = [] // friendly progress lines - private let manager = MCPServerManager() // The field is free text, so this app can't pin everything ahead of time. Two layers instead: // - the default model is pinned to a commit this app shipped with (developer-vouched); // - any other model is pinned on first download (`MLXFilePinStore`, kept outside the model @@ -106,6 +105,8 @@ final class AIQLModel: ObservableObject { pinnedRevisions: [AIQLModel.defaultModelRepo: AIQLModel.defaultModelRevision], supplyChainPolicy: MLXSupplyChainPolicy(trustPolicy: MlxCommunityOnly()), pinStore: MLXFilePinStore()) + // The MCP server goes into `lab.mcp`, the lab's manager — an app that uses LocalLMLab never + // makes an `MCPServerManager` of its own. private lazy var lab = LocalLMLab(configuration: .init(providers: [mlx, SystemModelProvider()])) init() { @@ -152,7 +153,7 @@ final class AIQLModel: ObservableObject { // 1 — connect (triggers an OAuth browser sign-in automatically if the server needs one) stage = .connecting step("Connecting to \(serverURL.host ?? serverURL.absoluteString)…") - let connection = await manager.addServer(url: serverURL, displayName: serverURL.host ?? "MCP server") + let connection = await lab.mcp.addServer(url: serverURL, displayName: serverURL.host ?? "MCP server") guard case .success(let server) = connection else { if case .failure(let error) = connection { stage = .failed("Couldn't connect to that MCP server — \(Self.describe(error))") @@ -165,6 +166,9 @@ final class AIQLModel: ObservableObject { stage = .failed("Connected, but that server didn't offer any tools to get data from."); return } step("Connected — \(server.tools.count) data tool(s) available.") + // Only this server's tools go to the model: one from an earlier Go stays connected in + // lab.mcp otherwise. (disconnect, not removeServer, which would forget its sign-in.) + for id in lab.mcp.servers.keys where id != server.id { lab.mcp.disconnect(id) } // 2 — download the model on first use if case .notDownloaded = lab.models.availability(for: modelID) { @@ -189,29 +193,37 @@ final class AIQLModel: ObservableObject { stage = .running step("Working… this takes a few minutes.") let outcome = await FolderAccess.withFolderAccessAsync { root in - await self.runPipeline(prompt: prompt, root: root, serverTools: server.tools, serverID: server.id, modelID: modelID) + await self.runPipeline(prompt: prompt, root: root, serverTools: server.tools, serverID: server.id) } stage = outcome ?? .failed("Couldn't open the folder you chose — pick it again.") } - private func runPipeline(prompt: String, root: URL, serverTools: [MCPToolDescriptor], serverID: MCPServerID, modelID: ModelID) async -> Stage { - // Wrap the server's tools as file-backed so the model can `saveAs` any of them. Cap the - // count — a small model degrades past ~8 tools; prefer names that look like "get a - // dataset" over admin/overview tools. + private func runPipeline(prompt: String, root: URL, serverTools: [MCPToolDescriptor], serverID: MCPServerID) async -> Stage { + // Offer the server's data tools from lab.mcp. Cap the count — a small model degrades past + // ~8 tools; prefer names that look like "get a dataset" over admin/overview tools. A new + // server's tools start disabled, so enable the chosen ones (and only those). let ranked = serverTools.sorted { lhs, rhs in Self.dataLikelihood(lhs.name) > Self.dataLikelihood(rhs.name) } - let dataTools: [any Tool] = ranked.prefix(4).compactMap { - try? FileBackedTool.mcp(descriptor: $0, manager: manager, root: root, inlineCharacterLimit: 8_000, - followUp: "load it into a table with loadTable, then query it with one sqlQuery") + let chosen = ranked.prefix(4).map(\.name) + for tool in serverTools { + lab.mcp.setToolEnabled(server: serverID, tool: tool.name, enabled: chosen.contains(tool.name)) } - guard !dataTools.isEmpty else { return .failed("Couldn't read that server's tools — its data format isn't supported yet.") } - - var tools: [any Tool] = dataTools - tools.append(LoadTableTool(root: root)) - tools.append(SQLQueryTool(root: root)) - tools.append(DescribeJSONTool(root: root)) - tools.append(CSVInfoTool(root: root)) - - let dataToolNames = dataTools.map(\.name).joined(separator: ", ") + // File-backed: each data tool gets a `saveAs` argument that writes its raw result into the + // chosen folder, so the dataset never enters the model's context. Only for this run — + // the folder's security-scoped access ends with it. + lab.mcp.setFileBackedOutput( + MCPFileBackedOutput(root: root, inlineCharacterLimit: 8_000, + followUp: "load it into a table with loadTable, then query it with one sqlQuery"), + server: serverID) + defer { lab.mcp.setFileBackedOutput(nil, server: serverID) } + + let tools: [any Tool] = [ + LoadTableTool(root: root), + SQLQueryTool(root: root), + DescribeJSONTool(root: root), + CSVInfoTool(root: root), + ] + + let dataToolNames = chosen.joined(separator: ", ") let instructions = """ You answer a data question by loading the pulled data into tables and running ONE SQL \ query. You never write row data yourself. @@ -243,7 +255,9 @@ final class AIQLModel: ObservableObject { // template toggle; the pipeline is mechanical (pick a tool, name a table, write // one SELECT) so the reasoning trace buys nothing but latency. session = try lab.makeSession(route: .local, tools: tools, instructions: instructions, - includeMCPTools: false, options: SessionOptions(effort: .off)) + includeMCPTools: true, options: SessionOptions(effort: .off)) + } catch LocalLMLabError.mcp { + return .failed("Couldn't read that server's tools — its data format isn't supported yet.") } catch { return .failed("Couldn't start the model: \(error.localizedDescription)") } From 6bf6480ca23924eb0579147a3ae23e2125aadeda Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:45:40 -0700 Subject: [PATCH 03/73] docs: 2.0 migration guide and CHANGELOG entry (skeleton for the MCP protocol parts) - docs/migrating-to-2.0.md: what breaks (per-server tool authorization with the 1.0 -> 2.0 mappings, two new MCP enum cases, MCPServerState decoding), what behaves differently (sessions follow lab.mcp, trusted servers ask only before destructive tools, app-only tools hidden, widget answers remembered), the one-declaration APIs, and what's new (chat-app pieces, MCP Apps / MCPAppsHost, per-server trust). - CHANGELOG: 2.0.0 (unreleased) entry with the same structure. - MCP protocol 2026-07-28 items are left as marked placeholders for the MCP protocol session to fill. Co-Authored-By: Claude --- CHANGELOG.md | 93 +++++++++++++++++++++++++++++++++ docs/migrating-to-2.0.md | 110 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 203 insertions(+) create mode 100644 docs/migrating-to-2.0.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 5336385..a4d30f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,99 @@ recompiling. A breaking change waits for 2.0. (Before RC.1, `1.0.0-beta.N` and e beta.4 → RC.1 changes that can break a build are listed under *Changed — breaking (pre-GA)* in the RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) +## 2.0.0 — unreleased + +**A one-time breaking release.** From `2.0.0`, every 2.x release is source compatible with it +(additive changes only, the same policy 1.x followed). Upgrading: [`docs/migrating-to-2.0.md`](docs/migrating-to-2.0.md). + +### Changed — breaking + +- **Tool authorization is decided per MCP server, by one authorizer.** `RuleBasedToolAuthorizer` + is removed; so are `ConfirmingToolAuthorizer.Requirement` and + `init(channel:timeout:requirement:)` — use `init(channel:policy:hostTools:timeout:)` with a + `ToolPolicy`. `MCPServerManager.setTrustsToolAnnotations(_:server:)` / + `trustsToolAnnotations(server:)` are replaced by `setTrust(_:server:)`. Mappings in the + migration guide §3. +- **New cases** on `MCPProtocolVersion` (`.v2026_07_28`) and `MCPConnectionMode` + (`.stateless(_:)`); a `switch` needs `@unknown default` (Swift 6 already requires it). +- **`MCPServerState`** gained fields (`trust`, `toolApproval`, `trustedToolsDigest`, + `liveUpdates`, …): a state JSON encoded under 1.0 no longer decodes. Restore servers with + `restore(from:)`. +- **One declaration each** where 1.x had kept two (usual call sites compile unchanged): + `makeSession` (now with `restoring:` and `mcpAppHints:`), `respond` / `streamResponse` (with + `options:` and `fromAppInstance:`), `LocalLMLab.Configuration.init` (with `mcp: MCPSettings`), + `PendingToolCall` / `PendingToolCallSummary` initializers (with `initiator:` and + `serverApproval:`). + +### Changed — behavior + +- A session's MCP tools follow `lab.mcp` between turns (1.0: fixed at `makeSession`). For a fixed + set, pass the tools in `tools:` with `includeMCPTools: false`. +- A trusted server asks only before destructive tools by default (1.0 asked before every change). +- MCP tools marked app-only (`_meta.ui.visibility` without `"model"`) are no longer given to + models; `ui://` resources are left out of model-facing resource lists. +- `languageModelSession` is documented as the escape hatch: run turns with `respond` / + `streamResponse`. + + + +### Added — building a chat app (`LocalLMLabSDKCore`) + +- `LocalLMLabSession.hostTranscript` (`HostTranscript`): the conversation as a UI shows it — the + user's message at turn start, tool calls as live `ToolCallRecord`s with their full MCP result + and app link, the reply and the model's reasoning (an inline `` block is split out), in + model order, never shortened by compaction; `replyInProgress` while streaming; + `archive()` / `restore(from:)`; a memory budget with spill to disk. +- `streamResponse(to:options:fromAppInstance:)`; `turnContext` (host text sent with every turn, + e.g. the date); `makeSession(…, restoring:)` to continue a saved conversation. +- `LocalLMLab.Configuration.mcp: MCPSettings` — how the lab sets up `lab.mcp`, the one MCP + manager. + +### Added — MCP Apps (`LocalLMLabSDKMCPAppsHost`, new, open source) + +- Shows an MCP server's interactive view (`ui://`, MCP Apps) when the model calls its tool: + `MCPAppViewController` / `MCPAppView` (sandboxed `WKWebView`), the host↔view bridge with an + explicit policy, `MCPAppsSessionBackend` (a view's tool calls go through the session's + authorizer, `ToolCallInitiator.app`), `MCPAppWidgetCache`, `MCPAppViewPool`, + `MCPAppRecreation`; `MCPClientHandlers.advertisingMCPApps()`. +- Core MCP client: `_meta` on tool, resource and result types (`MCPToolDescriptor.meta`, + `MCPResourceDescriptor.meta`, `MCPToolResult.meta`), `MCPToolAnnotations`, and + `MCPResponseLimits.maxUIResourceBytes` (cap on a view's HTML, default 4 MB). +- Core: `MCPAppLink`, `MCPAppVisibility`, `MCPToolDescriptor.app`, `isMCPAppResource`; + `LocalLMLabSession.callMCPTool(server:tool:arguments:initiator:)`, + `updateModelContext(_:fromAppInstance:)`; `makeSession(…, mcpAppHints:)`. +- `ConfirmingToolAuthorizer` remembers a view's answer per tool (a polling dashboard asks once). + +### Added — per-server trust and tool approval + +- `MCPServerTrust`, `ToolApproval` (+ `default(for:)`), `ToolApprover`, `ToolPolicy`, + `ToolPolicyDecision`; `MCPServerManager.setTrust(_:server:)`, `setToolApproval(_:server:)`, + `toolApproval(server:)`; `MCPServerState.trust` / `toolApproval` / `toolsChangedSinceTrusted`. +- `Components`: `MCPServerPickerView` shows each server's trust and approval, a notice when none of + a server's tools are on, and one when a trusted server's tools changed; the auth type is chosen + with a radio group. + +### Added — MCP protocol `2026-07-28` + + + +_To be written._ + +### Added — examples + +- [`mcp-chat`](examples/mcp-chat/): a chat with a local model (Qwen3 8B via MLX, or Apple's + on-device model) that shows MCP Apps views inline, with streaming, per-server trust and + approval, the date every turn, and conversations that reopen. + +### Checksums (SHA-256) + +_Filled in at release._ + +--- + ## 1.0.0 — GA — 2026-09-27 Fixes only; no API change. diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md new file mode 100644 index 0000000..da8f33f --- /dev/null +++ b/docs/migrating-to-2.0.md @@ -0,0 +1,110 @@ +# Migrating to the LocalLM Lab SDK 2.0 + +2.0 is a **one-time breaking release**: every 2.x release after it is source compatible with +`2.0.0`, additive changes only until 3.0. It breaks for two reasons. Several APIs had grown a +second way to do the same thing; 2.0 keeps one. And tool authorization moved to where trust +actually lives, the MCP server. + +Most apps change little. The usual `makeSession`, `respond`, `LocalLMLab.Configuration` and +`PendingToolCall` call sites compile unchanged. What you edit: tool authorization (§3), and a +`switch` over two MCP enums (§4). What behaves differently is in §5. + +## 1. Platform: unchanged + +`platforms: [.macOS("26.0")]`, Xcode 27 to build, macOS-27-only providers behind +`if #available(macOS 27, *)` — as in 1.0 ([`sdk-guide.md` §1a](sdk-guide.md)). + +## 2. Point your manifest at the new release + +Add `2.0.0` to your manifest's `knownSDKReleases` with the checksums from the release's +`.sha256` assets (tag `v2.0.0`), and build with: + +```sh +LOCALLM_SDK_VERSION=2.0.0 swift build +``` + +## 3. Tool authorization: one authorizer, decided per server + +In 1.0 an app wrote its policy as a closure (`ConfirmingToolAuthorizer`'s `requirement:`) or a +rule list (`RuleBasedToolAuthorizer`). In 2.0 the **MCP server** is the unit of trust, and you +state who may approve what where the tools come from: + +```swift +lab.mcp.setTrust(.trusted, server: companyServer) // believe its read-only / destructive labels +lab.mcp.setToolApproval(.allow, server: dashboardServer) // never ask for this one +let authorizer = ConfirmingToolAuthorizer(channel: confirmations) // the one authorizer +``` + +| 1.0 | 2.0 | +|---|---| +| `RuleBasedToolAuthorizer` (removed) | `ConfirmingToolAuthorizer` + per-server settings. `.confirm(atOrAbove: x)` → `hostTools: .ask(atOrAbove: x, by: .user)` for your own tools and `lab.mcp.setToolApproval(.ask(atOrAbove: x, by: .user), server:)` per server; `.confirmMCPTools` → nothing (an untrusted server asks by default); `.denyTool` / `.denyMCPTools` / `.deny(atOrAbove:)` / `.allowTool` → a `policy:` returning `.deny` / `.allow`, with those approvals routed `by: .policy`; the `confirm:` closure → a `ToolConfirmationChannel`. | +| `ConfirmingToolAuthorizer(channel:timeout:requirement:)` and `Requirement` (removed) | `ConfirmingToolAuthorizer(channel:policy:hostTools:timeout:)`. Move the closure into `policy:` (now `async`): `.allow` → `ToolPolicyDecision.allow`, `.confirm` → `.askUser`, `.deny(reason:)` → `.deny(reason:)`. To send every call to it: `hostTools: .ask(atOrAbove: .read, by: .policy)` and, per server, `setToolApproval(.ask(atOrAbove: .read, by: .policy), server:)`. Then move what settings can say (per-server "ask" / "don't ask") out of the policy. | +| `setTrustsToolAnnotations(true, server:)` / `trustsToolAnnotations(server:)` (removed) | `setTrust(.trusted, server:)`; `false` → `.untrusted`. Read it back from `MCPServerState.trust`. | +| `ConfirmingToolAuthorizer(channel:)` | Unchanged; the channel may now be `nil` for a host with no one to ask (calls that need a person are denied). | + +The defaults match 1.0's default rule (ask before anything that may change data), with one +difference for trusted servers — see §5. Save each server's `trust` and `toolApproval` with your +server list (the manager doesn't persist them) and set them again after `restore(from:)`. +`MCPServerState` gained fields, so a state JSON you encoded yourself under 1.0 won't decode; +restore servers through `restore(from:)` instead. Full walkthrough: +[`sdk-guide.md` §7c](sdk-guide.md). + +## 4. New cases on two MCP enums + +`MCPProtocolVersion` gained `.v2026_07_28` and `MCPConnectionMode` gained `.stateless(_:)`. Both +are non-frozen; Swift 6 already requires `@unknown default` in a `switch` over them, so this only +affects code that silenced that error another way. + +## 5. What behaves differently + +- **A session's MCP tools follow `lab.mcp` between turns.** Turn a tool on, add a server, or + change a server's trust, and the model has it on its next `respond` / `streamResponse` — same + session, same conversation. 1.0 fixed them at `makeSession`. To keep a session on a fixed set of + MCP tools, pass them in `tools:` with `includeMCPTools: false`. +- **A trusted server asks only before destructive tools** by default (1.0 asked before every + change regardless of trust). To keep the 1.0 behavior: + `lab.mcp.setToolApproval(.ask(atOrAbove: .mutate, by: .user), server:)`. +- **Tools an MCP server marks as only for its interactive view** (`_meta.ui.visibility` without + `"model"`) are no longer offered to models, and `ui://` resources are left out of model-facing + resource lists. +- **A widget's tool calls** (an MCP App view, §6) that need the user's answer ask once per view and + tool; destructive calls and timeouts are asked again. Model calls are unchanged. + +### MCP protocol + + + +_To be written with the MCP protocol `2026-07-28` documentation._ + +## 6. One way to do each thing + +These compile unchanged in their usual form; they are listed so you know which declaration is the +one to use (and what to edit if your code referred to an old overload as a function value, or +passed `restoring:` before `tools:`). + +| What | 2.0 | +|---|---| +| Make a session | One `makeSession(route:tools:instructions:restoring:includeMCPTools:mcpAppHints:options:authorizer:)`. Continue a saved conversation with `restoring:` (instead of `instructions:` — passing both traps). | +| Run a turn | `session.respond(to:options:fromAppInstance:)` / `streamResponse(to:options:fromAppInstance:)`. `languageModelSession` is the escape hatch: a turn run on it directly skips the chat history, `turnContext`, widget context, the MCP tool refresh and `retryOnContextOverflow`. | +| Set up MCP | `lab.mcp`, configured with `LocalLMLab.Configuration(providers:state:mcp:)` and `MCPSettings` (handlers, response limits, version negotiation, live updates). An app that uses `LocalLMLab` doesn't make its own `MCPServerManager`; making one directly is for apps without `LocalLMLab`. | +| Describe a pending call | One `PendingToolCall` / `PendingToolCallSummary` initializer each; `initiator:` defaults to `.model`, `serverApproval:` to `nil`. | + +## 7. What's new (opt in when you want it) + +- **Building a chat app.** `session.hostTranscript` is the conversation as your UI shows it: the + user's message as soon as it's sent, every tool call as a live record with its full result, the + reply (`replyInProgress` while streaming) and the model's reasoning split from its answer. Plus + `streamResponse(to:)`, `turnContext` (send the date and time with every message — a model has no + clock), and saving and reopening a conversation with the model's memory of it + (`makeSession(…, restoring:)`, `hostTranscript.archive()` / `restore(from:)`). +- **MCP Apps.** Some MCP servers ship an interactive view with a tool — Todoist's task list. The + new, open-source `LocalLMLabSDKMCPAppsHost` package shows it in the conversation when the model + calls that tool, sandboxed, with the view's own tool calls going through your authorizer. + Declare support with `MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps())`. + Reference app: [`examples/mcp-chat`](../examples/mcp-chat/). +- **Per-server trust and tool approval** (§3), shown on each row of `Components`' + `MCPServerPickerView`, which now also says when none of a server's tools are on. +- **MCP protocol `2026-07-28`** + _(see [`sdk-guide.md` §3a](sdk-guide.md))._ From 46f4373408d3a14fc79bc1b6134e6fe7b0751596 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:46:54 -0700 Subject: [PATCH 04/73] docs: MCP 2026-07-28 client support in the SDK guide, migration guide and CHANGELOG MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit sdk-guide §3a/§3c and the §12 MCP reference still described a 2025-11-25 ceiling and a handshake-only connection mode. Documents stateless mode, server/discover negotiation (MCPVersionNegotiation via MCPSettings), MRTR, live updates (MCPLiveUpdates, listen(filter:)), the dropped 2024-11-05 support, and the new diagnostics lines. --- CHANGELOG.md | 39 ++++++++++--- docs/mcp-diagnostics.md | 5 +- docs/migrating-to-2.0.md | 37 +++++++++--- docs/sdk-guide.md | 118 ++++++++++++++++++++++++++++++++------- 4 files changed, 164 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a4d30f0..02b7a26 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,8 +58,15 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) - `languageModelSession` is documented as the escape hatch: run turns with `respond` / `streamResponse`. - +- MCP servers that only speak `2024-11-05`, or use the old HTTP+SSE transport, fail `addServer` + with `.protocolMismatch` (1.x connected to the former and failed on the latter with "HTTP 404"). + `MCPProtocolVersion.v2024_11_05` is deprecated and kept so stored values decode. +- Each connect first probes `server/discover` (`versionNegotiation: .auto`, the default) and falls + back to `initialize` when a server rejects it. `.legacy` skips the probe. +- Elicitation is declared as `{"form":{},"url":{}}` from `2025-11-25` (was `{}`, which strict + servers read as form-only). +- `MCP-Protocol-Version` is sent only when the negotiated revision is `2025-06-18` or later, and + an inbound `elicitation/create` is refused below `2025-06-18`. ### Added — building a chat app (`LocalLMLabSDKCore`) @@ -99,11 +106,29 @@ HTTP+SSE fails cleanly, elicitation capability shape, versionNegotiation .auto). ### Added — MCP protocol `2026-07-28` - - -_To be written._ +- The MCP client speaks `2026-07-28`, the current revision, and negotiates down to `2025-03-26` + (`MCPProtocolVersion.v2026_07_28`, `.statelessPreferred`, `.minimumSupported`, `.isStateless`). +- **Stateless mode** (`MCPConnectionMode.stateless(_:)`): `server/discover` instead of + `initialize`, no session, the per-request `_meta` envelope, and `Mcp-Method` / `Mcp-Name` / + `Mcp-Param-*` headers. +- **`MCPVersionNegotiation`** (`.auto` / `.legacy` / `.pin(_:)`), set with + `MCPSettings(versionNegotiation:)` or `MCPServerManager(versionNegotiation:)`. +- **Multi-round-trip requests (MRTR).** A request that comes back `input_required` is answered + through the existing `MCPClientHandlers` (elicitation in form or URL mode, sampling, roots) and + re-sent with the answers, for up to 8 rounds. No handler API changes. +- **Live updates** (`subscriptions/listen`). One stream per stateless server that announces list + changes; tools, prompts and resources are re-listed on change with enabled flags kept. + `MCPServerState.liveUpdates: MCPLiveUpdates?` (`.active` / `.reconnecting` / `.unavailable`). + Opt out with `MCPSettings(liveUpdates: false)` or `MCPServerManager(liveUpdates: false)`. Custom + transports: `MCPConnection.listen(filter:)` with `MCPSubscriptionFilter`, `MCPChangeEvent`, + `MCPListenEnd` (default implementation ends as refused). +- `MCPProtocolVersion.supportsURLElicitation` and `.sendsProtocolVersionHeader` feature gates. +- Diagnostics: the `server/discover` probe and fallback, live-update status, and MRTR rounds + (export area `mrtr`) are logged ([`docs/mcp-diagnostics.md`](docs/mcp-diagnostics.md)). +- Authorization fixes found by the official MCP conformance suite: protected-resource metadata + at the path-suffixed then root well-known URL; `resource` must cover the server URL; metadata + `issuer` and redirect `iss` are checked (RFC 8414, RFC 9207); a changed authorization server + mid-connection drops the old credentials and signs in again. ### Added — examples diff --git a/docs/mcp-diagnostics.md b/docs/mcp-diagnostics.md index be6e17f..4925f71 100644 --- a/docs/mcp-diagnostics.md +++ b/docs/mcp-diagnostics.md @@ -111,8 +111,9 @@ a glance before sending it on. | Category | What you'll find there | |---|---| -| `MCP.connection` | the protocol version offered and agreed, the server's name and capabilities; every method call (at `debug`); any non-2xx HTTP response with its content type; a 401 and the start of sign-in (with the `WWW-Authenticate` header); a 403 `insufficient_scope` and the scope step-up; a session id being issued; a personal access token being rejected | -| `MCP.manager` | `addServer` and its result (agreed version, tool count); a tool result whose structured data didn't match the tool's declared output schema | +| `MCP.connection` | the `server/discover` probe and its outcome — negotiated stateless, rejected and falling back to `initialize`, no common version, or a sign-in failure (which does not fall back); the protocol version offered and agreed, the server's name and capabilities; a server on the unsupported `2024-11-05` HTTP+SSE transport; a stateless request retried at another version after "unsupported protocol version"; a `subscriptions/listen` stream opening and ending (with the reason); every method call (at `debug`); any non-2xx HTTP response with its content type; a 401 and the start of sign-in (with the `WWW-Authenticate` header); a 403 `insufficient_scope` and the scope step-up; a session id being issued; a personal access token being rejected | +| `MCP.manager` | `addServer` and its result (agreed version, tool count); live updates becoming active (what the server granted), unavailable (why), or resuming after sign-in; a live update re-listing tools; a tool result whose structured data didn't match the tool's declared output schema | +| `MCP.mrtr` *(export only)* | `2026-07-28` multi-round-trip requests: a call coming back `input_required` (which inputs it asked for); an input the client can't answer (no handler registered, or elicitation suppressed for that call); a call stopped for exceeding the 8-round limit. These lines are labelled `MCP.mrtr` in `exportText()` / `exportJSON()`, but in Console and `log stream` they appear under the `MCP.connection` category. | | `MCP.oauth` | the protected-resource metadata fetch; which `.well-known` discovery URL resolved (or that none did); the choice between Dynamic Client Registration, CIMD, and a manual client id — and why CIMD was skipped if it was; the PKCE check; the browser being opened (endpoint, `client_id`, scope); an `iss` mismatch; the code coming back; the token-exchange HTTP status; whether a silent refresh worked; every failure with the server's actual (truncated, redacted) response | | `MCP.stream` | each server-sent-event frame as it's classified — an inbound request (`elicitation/create` and the like), a notification, a result; a server error result; a stream that broke mid-request; a stream that ended without the result it was waiting for | diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md index da8f33f..95ae092 100644 --- a/docs/migrating-to-2.0.md +++ b/docs/migrating-to-2.0.md @@ -72,11 +72,29 @@ affects code that silenced that error another way. ### MCP protocol - - -_To be written with the MCP protocol `2026-07-28` documentation._ +The client now speaks MCP `2026-07-28` and falls back to the `2025` handshake on its own +([`sdk-guide.md` §3a](sdk-guide.md#3a-which-mcp-revision-the-client-speaks--and-why-you-mostly-dont-have-to-care)). +Code needs no changes. What a user may notice: + +- **`2024-11-05` servers are refused.** `addServer` fails with `.protocolMismatch`, where 1.x + connected. Most servers of that era use the old two-endpoint HTTP+SSE transport, which the + client never supported; 1.x failed on them with a confusing "HTTP 404". 2.0 detects that + transport with one `GET` and reports `.protocolMismatch` instead. The oldest supported revision + is `MCPProtocolVersion.minimumSupported` (`2025-03-26`). +- **`server/discover` is sent first.** With the default `versionNegotiation: .auto`, every + connect probes for `2026-07-28` before `initialize`. A server on an older revision rejects the + probe and the client falls back, so this is one extra request per connect. If a server + mishandles the probe, use `MCPSettings(versionNegotiation: .legacy)` to skip it. +- **Elicitation is declared as `{"form":{},"url":{}}`** from `2025-11-25`, not `{}`. Strict + servers read `{}` as form-only and refused URL-mode elicitation, so URL mode now works on + them. Nothing to change unless you inspect the wire. +- **Tool lists can change mid-connection.** A `2026-07-28` server that announces changes gets + them re-listed automatically, and `serverChanges` fires. A new tool arrives disabled; enabled + flags on existing tools are kept. If your UI assumes a fixed list after `addServer`, either + handle the update or set `MCPSettings(liveUpdates: false)`. +- **`allowElicitation: false`** on a `2026-07-28` server leaves elicitation out of the request's + capabilities, and a server that requires it fails the call (`.serverError`). On a `2025` + server the same call still gets a decline. ## 6. One way to do each thing @@ -106,5 +124,10 @@ passed `restoring:` before `tools:`). Reference app: [`examples/mcp-chat`](../examples/mcp-chat/). - **Per-server trust and tool approval** (§3), shown on each row of `Components`' `MCPServerPickerView`, which now also says when none of a server's tools are on. -- **MCP protocol `2026-07-28`** - _(see [`sdk-guide.md` §3a](sdk-guide.md))._ +- **MCP protocol `2026-07-28`**, the current revision. It is stateless: no session, no + handshake, and requests that gateways can route by header. A tool that needs the user's input + mid-call reaches your existing `MCPElicitationHandler` through multi-round-trip requests. A + server can push tool, prompt and resource list changes, and the client applies them without a + reconnect (`MCPServerState.liveUpdates` shows the status). All automatic; older servers keep + working through the `2025` handshake. See + [`sdk-guide.md` §3a](sdk-guide.md#3a-which-mcp-revision-the-client-speaks--and-why-you-mostly-dont-have-to-care). diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 833739d..c3d0175 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -418,25 +418,63 @@ instead of yours. > negotiates a revision automatically; none of them reads `negotiatedProtocolVersion` directly in > normal use. -MCP is versioned by date-stamped revisions (`2024-11-05`, `2025-03-26`, `2025-06-18`, -`2025-11-25`). You don't pick one. On `addServer` the client offers the newest it knows -(`MCPProtocolVersion.clientPreferred`, currently `2025-11-25`) and the server replies with the -newest *it* knows; they meet at the highest revision both support. An old server that only -speaks `2025-03-26` still connects and works — nothing you wrote for the `0.8.x` MCP client -breaks. +MCP is versioned by date-stamped revisions. The client supports `2025-03-26`, `2025-06-18`, +`2025-11-25` and **`2026-07-28`** (new in 2.0). You don't pick one. On `addServer` the client +first sends `server/discover`, the `2026-07-28` way of asking a server which revisions it speaks. +A server on the current revision answers, and the connection is **stateless**. Any older server +rejects that unknown method, and the client falls back to the `initialize` handshake, offering +`MCPProtocolVersion.clientPreferred` (`2025-11-25`). The two meet at the newest revision both +support. Nothing you wrote against the 1.x client changes. + +`2026-07-28` reworks the wire protocol, and the SDK absorbs all of it: + +- **No session and no handshake.** Every request carries the protocol version and client + capabilities itself, plus `Mcp-Method` / `Mcp-Name` headers that gateways can route on. +- **Server requests become retries (MRTR).** A tool that needs input mid-call no longer pushes + `elicitation/create` down an open stream. It returns "input required", and the client asks + your handler and re-sends the call with the answer. Your `MCPElicitationHandler` + ([§3c](#3c-server-initiated-requests-elicitation-and-the-sampling--roots-seams)) is called + exactly as before, on either revision. Multi-step flows are capped at 8 rounds. +- **Live updates.** A stateless server that advertises `listChanged` gets one + `subscriptions/listen` stream. When its tools, prompts or resources change, the manager + re-lists them on its own (enabled flags are kept; a new tool arrives disabled) and + `serverChanges` fires. No Reconnect needed. What the negotiated revision changes is how much a server *can* hand you: | Revision the server meets you at | What you get | |---|---| -| `2024-11-05` / `2025-03-26` | Tools, resources, prompts. Tool results are plain text. | +| `2025-03-26` | Tools, resources, prompts. Tool results are plain text. | | `2025-06-18` | + structured tool results (`MCPToolResult.structuredContent`), `resource_link` results, elicitation, tool `title`s | | `2025-11-25` | + icons, richer elicitation field types, URL-mode elicitation | +| `2026-07-28` | + stateless transport, elicitation via MRTR, live tool/prompt/resource updates | `MCPServerState.negotiatedProtocolVersion` tells you where a given connection landed (`nil` -until connected). You rarely read it — the point of the table is that the newer fields on -`MCPToolResult` and the elicitation seam below are simply empty / never-called against an older -server, not errors to guard. +until connected; `.isStateless` is true for `2026-07-28`). `MCPServerState.liveUpdates` is +`.active(…)`, `.reconnecting` or `.unavailable(reason)` for a stateless server and `nil` +otherwise. You rarely read either. The point of the table is that newer fields on +`MCPToolResult` and the elicitation seam below are simply empty or never called against an older +server. They are not errors to guard. + +**Servers the client refuses.** A `2024-11-05` server, or one on the old two-endpoint HTTP+SSE +transport, fails `addServer` with `.protocolMismatch`. Both were accepted in 1.x. This is the +one behaviour change in this area (see `migrating-to-2.0.md`). + +**Two knobs on `MCPSettings`, both rarely needed:** + +```swift +let mcp = MCPSettings( + versionNegotiation: .auto, // default. .legacy = never send server/discover (2025 handshake only); + // .pin(.v2026_07_28) = stateless only, no fallback + liveUpdates: true // default. false = no subscriptions/listen streams +) +// pass as LocalLMLab.Configuration(…, mcp: mcp). An app without LocalLMLab passes the same +// two arguments to MCPServerManager(versionNegotiation:liveUpdates:). +``` + +Use `.legacy` if a server mishandles the `server/discover` probe (for example, it hangs instead +of rejecting it). The client already falls back on a timeout, so this only saves the wait. Use +`liveUpdates: false` when your app shows a fixed tool list and must not change it mid-session. ### 3b. What a tool call gives you back: `MCPToolResult` @@ -533,6 +571,13 @@ URL-mode `url`) → `MCPElicitationResponse`. `callTool` also takes `allowElicit — pass `false` where there's no way to show a prompt (a background job) and the request is declined cleanly instead of hanging. +On a `2026-07-28` server the same handlers are reached through MRTR rather than an in-stream +request ([§3a](#3a-which-mcp-revision-the-client-speaks--and-why-you-mostly-dont-have-to-care)), +and nothing in your code changes. The one visible difference: with `allowElicitation: false` +the client leaves elicitation out of that request's declared capabilities. A server that +requires it then fails the call outright (`.serverError`), where a `2025-11-25` server would +have received a decline. + **Sampling** has a security edge: a registered `MCPSamplingHandler` lets a server push a prompt through your model. Your handler owns the human-in-the-loop approval; the SDK gives you the seam, not the policy. Don't register it if you don't need it. @@ -3125,7 +3170,8 @@ enum GenerationErrorDescription { ```swift final class MCPServerManager { - init(responseLimits: MCPResponseLimits = .default, handlers: MCPClientHandlers = .init()) // NOT a singleton + init(responseLimits: MCPResponseLimits = .default, handlers: MCPClientHandlers = .init(), + versionNegotiation: MCPVersionNegotiation = .auto, liveUpdates: Bool = true) // NOT a singleton private(set) var servers: [MCPServerID: MCPServerState] { get } var serverChanges: AsyncStream<[MCPServerID: MCPServerState]> { get } var estimatedTotalTokens: Int { get } @@ -3174,17 +3220,42 @@ struct MCPServerState: Codable, Sendable { var resourceTemplates: [MCPResourceTemplateDescriptor] var prompts: [MCPPromptDescriptor] var negotiatedProtocolVersion: MCPProtocolVersion? // nil until connected ([§3a](#3a-which-mcp-revision-the-client-speaks--and-why-you-mostly-dont-have-to-care)) - var serverInstructions: String? // server's own usage note from `initialize` + var serverInstructions: String? // server's own usage note (initialize or server/discover) + var liveUpdates: MCPLiveUpdates? // stateless servers only; nil otherwise ([§3a](#3a-which-mcp-revision-the-client-speaks--and-why-you-mostly-dont-have-to-care)) func exportSummary() -> String // plain-text listing of tools/resources/prompts, enabled state, token cost } enum MCPProtocolVersion: String, CaseIterable, Comparable, Sendable, Codable { - case v2024_11_05 = "2024-11-05", v2025_03_26 = "2025-03-26" - case v2025_06_18 = "2025-06-18", v2025_11_25 = "2025-11-25" - static let clientPreferred: MCPProtocolVersion = .v2025_11_25 - var supportsStructuredContent: Bool { get } // >= 2025-06-18 - var supportsElicitation: Bool { get } + @available(*, deprecated) case v2024_11_05 = "2024-11-05" // no longer negotiated; kept so stored values decode + case v2025_03_26 = "2025-03-26", v2025_06_18 = "2025-06-18" + case v2025_11_25 = "2025-11-25", v2026_07_28 = "2026-07-28" + static let clientPreferred: MCPProtocolVersion = .v2025_11_25 // offered at initialize (handshake era) + static let statelessPreferred: MCPProtocolVersion = .v2026_07_28 // probed with server/discover + static let minimumSupported: MCPProtocolVersion = .v2025_03_26 // lower → .protocolMismatch + var isStateless: Bool { get } // >= 2026-07-28 + var supportsStructuredContent: Bool { get } // >= 2025-06-18 + var supportsElicitation: Bool { get } // >= 2025-06-18 var supportsIconsAndRicherElicitation: Bool { get } // >= 2025-11-25 + var supportsURLElicitation: Bool { get } // >= 2025-11-25 + var sendsProtocolVersionHeader: Bool { get } // >= 2025-06-18 +} + +// How connect() chooses between the stateless and handshake models (mirrors the TS SDK's option). +enum MCPVersionNegotiation: Sendable, Equatable { + case auto // default: server/discover, fall back to initialize + case legacy // initialize only + case pin(MCPProtocolVersion) // stateless at this revision, no fallback +} + +// Live updates (2026-07-28 subscriptions/listen). The manager runs these for you; read +// MCPServerState.liveUpdates for status. Opt out with MCPServerManager(liveUpdates: false). +enum MCPLiveUpdates: Sendable, Codable, Equatable { + case active(MCPSubscriptionFilter) // listening; what the server granted + case reconnecting // stream dropped, retrying with backoff + case unavailable(String) // refused or sign-in expired; resumes on next connect +} +struct MCPSubscriptionFilter: Sendable, Codable, Equatable { + var toolsListChanged: Bool; var promptsListChanged: Bool; var resourcesListChanged: Bool } enum MCPAuthType: String, Codable, Sendable { case none, pat, oauthManual } @@ -3355,14 +3426,23 @@ struct MCPServerCapabilities: Sendable, Codable, Equatable { var logging: Bool; var completions: Bool var hasExperimental: Bool // server declared something in `experimental`, kept opaque } -// How the connection talks to the server. Only .handshake exists today (every revision through -// 2025-11-25); 2026-07-28's stateless model adds a second case later without changing callers. +// How the connection talks to the server: initialize + session (through 2025-11-25), or +// 2026-07-28's stateless model (server/discover, per-request _meta, MRTR). Callers of +// MCPServerManager never see the difference. enum MCPConnectionMode: Sendable, Equatable { case handshake(MCPProtocolVersion) + case stateless(MCPProtocolVersion) var negotiatedVersion: MCPProtocolVersion { get } + var isStateless: Bool { get } } ``` +A custom `MCPConnection` also inherits `listen(filter:) -> AsyncStream` with a +default that ends at once as `.refused`. Override it only if your transport supports change +notifications. `MCPChangeEvent` is `.acknowledged(filter)`, `.toolsChanged`, `.promptsChanged`, +`.resourcesChanged`, then a final `.ended(MCPListenEnd)`, where `MCPListenEnd` is `.closed`, +`.failed(String)`, `.refused(String)` or `.unauthorized`. + **Credential presence, without a Keychain prompt.** A host that lets a user swap the entire MCP server list in one shot — a config-profile Load — needs to know, for each server about to be restored, whether reconnecting will need a fresh sign-in. A server that authenticates via From 4ca921dc8149cb6ecc56ce3828daf2251aed0193 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:49:52 -0700 Subject: [PATCH 05/73] docs: chat apps and MCP Apps in the SDK guide (2.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - sdk-guide.md: new §3f "MCP Apps: showing a server's interactive views" (declaring support, rendering with LocalLMLabSDKMCPAppsHost, a view's calls through the session's authorizer, the live-view pool and re-creation, what a view may do); new §6a subsection "Building a chat app" (hostTranscript, streamResponse, replyInProgress, turnContext, reopening a conversation); makeSession text for 2.0 (one function, tools follow lab.mcp, restoring:, mcpAppHints:, languageModelSession as the escape hatch); lab.mcp + MCPSettings as the one MCP manager. - §12 reference: Configuration(mcp:), MCPSettings, makeSession, LocalLMLabSession (turnContext, hostTranscript, respond/streamResponse, callMCPTool, updateModelContext), HostTranscript, ToolCallRecord, MCP Apps fields on MCP types, MCPClientHandlers.extensions, and a new "MCP Apps (LocalLMLabSDKMCPAppsHost)" block. - migrating-to-2.0.md links to the new sections. Co-Authored-By: Claude --- docs/migrating-to-2.0.md | 6 +- docs/sdk-guide.md | 306 +++++++++++++++++++++++++++++++++++++-- 2 files changed, 297 insertions(+), 15 deletions(-) diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md index 95ae092..505d90a 100644 --- a/docs/migrating-to-2.0.md +++ b/docs/migrating-to-2.0.md @@ -116,12 +116,14 @@ passed `restoring:` before `tools:`). reply (`replyInProgress` while streaming) and the model's reasoning split from its answer. Plus `streamResponse(to:)`, `turnContext` (send the date and time with every message — a model has no clock), and saving and reopening a conversation with the model's memory of it - (`makeSession(…, restoring:)`, `hostTranscript.archive()` / `restore(from:)`). + (`makeSession(…, restoring:)`, `hostTranscript.archive()` / `restore(from:)`). See + [`sdk-guide.md` "Building a chat app"](sdk-guide.md#building-a-chat-app--hosttranscript-streamresponse-turncontext). - **MCP Apps.** Some MCP servers ship an interactive view with a tool — Todoist's task list. The new, open-source `LocalLMLabSDKMCPAppsHost` package shows it in the conversation when the model calls that tool, sandboxed, with the view's own tool calls going through your authorizer. Declare support with `MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps())`. - Reference app: [`examples/mcp-chat`](../examples/mcp-chat/). + Reference app: [`examples/mcp-chat`](../examples/mcp-chat/); walkthrough + [`sdk-guide.md` §3f](sdk-guide.md#3f-mcp-apps-showing-a-servers-interactive-views). - **Per-server trust and tool approval** (§3), shown on each row of `Components`' `MCPServerPickerView`, which now also says when none of a server's tools are on. - **MCP protocol `2026-07-28`**, the current revision. It is stateless: no session, no diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index c3d0175..676ec28 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -638,6 +638,64 @@ Full detail — every log category, what `MCPDiagnostics.observer` gives you, th guarantees, and the end-to-end support flow — is in [`mcp-diagnostics.md`](mcp-diagnostics.md). +### 3f. MCP Apps: showing a server's interactive views + +> **Reach for this when** a server you connect to ships a view with its tools (Todoist's task +> list, a dashboard) and you want it in your app's conversation. +> +> **Examples that use it:** [`mcp-chat`](../examples/mcp-chat/). + +An **MCP App** is a small interactive view (HTML, `ui://` resource) a server provides alongside a +tool. The model never asks for a view: it picks a tool as usual, and if that tool's listing names a +view (`MCPToolDescriptor.app`), your app shows it under the call once it finishes, handed the +call's arguments and result. Three pieces: + +1. **Say you can show views.** Servers only offer them to a client that declares support: + + ```swift + let lab = LocalLMLab(configuration: .init( + providers: [...], + mcp: MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps()))) // LocalLMLabSDKMCPAppsHost + let session = try lab.makeSession(route: .chat, mcpAppHints: true, authorizer: authorizer) + ``` + +2. **Render the view** for a finished call with `LocalLMLabSDKMCPAppsHost` (open source, a + separate package built on Core's public API): + + ```swift + import LocalLMLabSDKMCPAppsHost + + let cache = MCPAppWidgetCache() + let resource = try await cache.fetch(resourceURI: record.app!.resourceURI, server: server, + manager: lab.mcp, recordID: record.id) // checked, cached by hash + let controller = MCPAppViewController( + resource: resource, + tools: lab.mcp.servers[server]?.tools ?? [], + backend: MCPAppsSessionBackend(session: session, manager: lab.mcp, server: server, instance: record.id), + configuration: MCPAppsBridgeConfiguration(hostName: "My App", hostVersion: "1.0"), + actions: .session(session, instance: record.id)) // ui/message, ui/update-model-context → the session + MCPAppRecreation.deliver(record, to: controller) // the call's arguments and result + await controller.load() + // SwiftUI: MCPAppView(controller: controller).frame(height: controller.contentHeight ?? 360) + ``` + +3. **Keep many views in one conversation.** Each live view is a WebKit process (~100 MB). + `MCPAppViewPool` keeps a few live (`limit`, default 3) and snapshots the rest + (`snapshot(for:)`) until they're needed again. To show an old call again — after a relaunch, + say — `MCPAppRecreation.revalidate(record, source:cache:)` decides: the current view with the + stored result when the tool's output still has the same shape, `.needsRefresh` when it changed, + view-only from the cache when the server can't be reached. + +**What a view may do.** It runs in a sandboxed `WKWebView`: no network unless your +`MCPAppsSandboxPolicy` allows the hosts it declares, no camera, microphone or location. Its tool +calls go through the bridge's `MCPAppsBridgePolicy` and then — with `MCPAppsSessionBackend` — +through your session's authorizer as `ToolCallInitiator.app(instance:)`, recorded in +`hostTranscript` like the model's (tools the server marks app-only, such as a dashboard's polling, +are counted per view instead of recorded). Sending a message as the user (`ui/message`), sharing +context with the model (`ui/update-model-context`) and opening links are denied unless your +policy allows them. `ConfirmingToolAuthorizer` remembers the user's answer per view and tool, so a +polling view asks once (destructive calls ask each time). + ## 4. Keychain storage — automatic isolation, native API, sandbox-safe > **You don't reach for this — you get it for free.** There is no "set up credential storage" @@ -959,11 +1017,17 @@ let lab = LocalLMLab(configuration: .init(providers: [ // rest inside `if #available(macOS 27, *)` — see [§1a](#1a-targeting-macos-26-and-macos-27-from-one-build). lab.models // the @Observable ModelRegistry — providers, routes, residency, downloads -lab.mcp // MCPServerManager, unchanged from 0.8.x +lab.mcp // the app's one MCPServerManager — set up via Configuration(…, mcp: MCPSettings(…)), §3a lab.workspace // security-scoped workspace access + ready-made tools lab.connectors // Calendar / Reminders / Contacts / Location ``` +An app that uses `LocalLMLab` uses `lab.mcp` and never makes an `MCPServerManager` of its own: +`makeSession` takes MCP tools from `lab.mcp`, and a second manager would be invisible to it. +Configure it with `MCPSettings` (handlers, response limits, version negotiation, live updates — +the same settings `MCPServerManager`'s initializer takes, same defaults); making an +`MCPServerManager` directly is for apps that don't use `LocalLMLab`. + `lab.snapshot() -> LocalLMLabState` gives you a `Codable` snapshot of the route map + residency policy + installed-model records to persist wherever you like (the SDK writes nothing to disk); `lab.restore(from:)` re-applies one. Use it when you want the user's model choices to survive a @@ -1298,13 +1362,28 @@ let session = try lab.makeSession( tools: [SearchWorkspaceTool(), ApplyPatchTool(), myGitTool], instructions: "You are a coding assistant.", ) -// Drive the tool loop yourself — nothing here runs an agent loop: +// Run a turn — nothing here runs an agent loop: let answer = try await session.respond(to: task) -// or opt out of the SDK's retry wrapper and use Apple's session directly: -for try await snapshot in session.languageModelSession.streamResponse(to: task) { … } +// or stream it (each element is the reply so far): +for try await partial in session.streamResponse(to: task) { … } ``` -`session.route` / `session.modelID` tell you what actually backs it. +`session.route` / `session.modelID` tell you what actually backs it. `makeSession` is one +function, `makeSession(route:tools:instructions:restoring:includeMCPTools:mcpAppHints:options:authorizer:)`: + +- **MCP tools follow `lab.mcp`.** With `includeMCPTools` (the default) the session offers the + enabled MCP tools and keeps following `lab.mcp`: turn a tool on or off, add a server, or change + a server's trust, and the model has it on its next `respond` / `streamResponse` — same session. + For a fixed, per-session set of MCP tools, pass them in `tools:` with `includeMCPTools: false`. +- **`restoring:`** continues a saved conversation (see [Building a chat app](#building-a-chat-app--hosttranscript-streamresponse-turncontext)); + leave `instructions` nil then — the saved transcript carries its own (passing both traps). +- **`mcpAppHints: true`** adds one line to the description of each MCP tool that has an MCP App + view, so a model can prefer it when the user asks to *see* something ([§3f](#3f-mcp-apps-showing-a-servers-interactive-views)). + +**Run turns with `respond` / `streamResponse`.** They keep `hostTranscript` up to date, send +`turnContext` and any widget context, refresh MCP tools and apply `retryOnContextOverflow`. +`languageModelSession` is Apple's session underneath — the escape hatch for what the SDK doesn't +surface (the raw `transcript` to save, say); a turn run on it directly skips all of the above. **`options: SessionOptions`** carries per-call knobs — provider-native web search, sampling (`temperature` / `topP` / `topK` / `seed` / `maxOutputTokens`), and `effort`. `effort: .off` asks the model **not @@ -1351,11 +1430,72 @@ seed → usually the same text" as the contract; don't build tests or caching on If you call `session.languageModelSession.respond(...)` directly to opt out of the SDK's wrapper, these options are **not** applied — pass `try options.appleGenerationOptions()` yourself. +### Building a chat app — `hostTranscript`, `streamResponse`, `turnContext` + +> **Use it when** you're drawing a conversation: user bubbles, tool-call rows, the streaming +> reply, the model's reasoning — and want it to survive a relaunch. +> +> **Examples that use it:** [`mcp-chat`](../examples/mcp-chat/) — the whole chat UI is +> `session.hostTranscript` plus `streamResponse`. + +Apple's transcript is the model's context: it can be compacted, and it keeps only what the model +saw of a tool result. `session.hostTranscript` is the conversation as your UI shows it — +append-only, in the order the model worked, and `@Observable`, so a SwiftUI view that reads it +redraws as it changes: + +```swift +session.turnContext = { "[Current date and time: \(Date().formatted())]" } // sent with every turn, + // not shown in the transcript +for try await _ in session.streamResponse(to: text) {} // the view reads hostTranscript + +ForEach(session.hostTranscript.entries) { entry in + switch entry.content { + case .user(let text): UserBubble(text) // appears as the turn starts + case .assistant(let text): Reply(text) // the answer only + case .reasoning(let text): ReasoningDisclosure(text) // incl. Qwen3's block, split out + case .toolCall(let record): ToolCallRow(record) // live: .running → .finished + case .appContext(_, let text): ViewSharedContext(text) // an MCP App view told the model this + @unknown default: EmptyView() + } +} +if let reply = session.hostTranscript.replyInProgress { StreamingReply(reply) } // .reasoning / .text / .isReasoning +``` + +- **When it updates.** The user's message is added as the turn starts; each tool call appears + when it starts and changes in place as it finishes; the reply and reasoning land when the turn + ends (`replyInProgress` until then — cleared in the same update, so a view never shows both). + A turn run on `languageModelSession` directly gets text only after + `reconcileServerToolArtifacts()`. +- **Tool calls are full records** (`ToolCallRecord`): state, arguments, the output the model saw, + the full `MCPToolResult` (`mcpResult` — kept under a memory budget, spilled to disk past it), + the tool's MCP App link (`app`), who made the call (`initiator`: the model, an MCP App view, or + your app), and timestamps. +- **A model has no clock.** `turnContext` is called as each turn starts and its text goes ahead of + the user's message — current even in a restored conversation, whose instructions were written + long ago. (The ready-made `ClockTool` lets the model ask too, but a small model may not.) +- **Tools of a newly added MCP server start off.** Show the user that, or the model gets no tools + — and may claim to have used one anyway. + +**Reopening a conversation.** Save both halves and restore both: + +```swift +let shown = try session.hostTranscript.archive() // what the user saw +let memory = try JSONEncoder().encode(session.languageModelSession.transcript) // the model's memory +// next launch: +let saved = try JSONDecoder().decode(Transcript.self, from: memory) +let session = try lab.makeSession(route: .chat, restoring: saved) +try session.hostTranscript.restore(from: shown) +``` + +Restore only `shown` to show an old conversation while the model starts fresh. The restored +transcript's entries are history: never added to `hostTranscript` again. + ### `LocalLMLabSession.events` — the side-channel Apple's streaming doesn't give you > **Use it when** your UI needs to show what's happening *around* generation — a spinner per > tool call, a "compacting context…" notice. Token streaming stays on -> `languageModelSession.streamResponse`; `events` carries only the rest. +> `session.streamResponse`; `events` carries only the rest. (A chat UI can also read tool calls +> live from `hostTranscript` — next section.) > > **Examples that use it:** [`code-buddy`](../examples/code-buddy/)'s `→ tool` / `✓ tool` stderr > trace is this stream. @@ -2327,9 +2467,12 @@ the signature; the numbered sections above are the intended learning path. ```swift // --- front door --------------------------------------------------------------- @MainActor final class LocalLMLab { - struct Configuration { var providers: [any ModelProvider]; var state: LocalLMLabState? ; init(providers: [any ModelProvider] = [], state: LocalLMLabState? = nil) } + struct Configuration { + var providers: [any ModelProvider]; var state: LocalLMLabState?; var mcp: MCPSettings + init(providers: [any ModelProvider] = [], state: LocalLMLabState? = nil, mcp: MCPSettings = MCPSettings()) + } let models: ModelRegistry - let mcp: MCPServerManager + let mcp: MCPServerManager // the one manager, made from Configuration.mcp let connectors: ConnectorsFacade // .calendar/.reminders/.contacts/.location permission lifecycle let workspace: WorkspaceFacade init(configuration: Configuration = .init()) @@ -2337,10 +2480,22 @@ the signature; the numbered sections above are the intended learning path. func restore(from state: LocalLMLabState) // from LocalLMLab+makeSession: func makeSession(route: RouteName, tools: [any Tool] = [], instructions: String? = nil, - includeMCPTools: Bool = true, options: SessionOptions = .init(), + restoring transcript: Transcript? = nil, // continue a saved conversation (instructions nil) + includeMCPTools: Bool = true, // and keep following lab.mcp between turns + mcpAppHints: Bool = false, // [§3f](#3f-mcp-apps-showing-a-servers-interactive-views) + options: SessionOptions = .init(), authorizer: (any ToolCallAuthorizer)? = nil) throws -> LocalLMLabSession // authorizer: [§7c](#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first) } +struct MCPSettings { // how the lab sets up lab.mcp — MCPServerManager's own defaults + var handlers: MCPClientHandlers // incl. extensions, e.g. .advertisingMCPApps() (LocalLMLabSDKMCPAppsHost) + var responseLimits: MCPResponseLimits + var versionNegotiation: MCPVersionNegotiation // .auto + var liveUpdates: Bool // true + init(handlers: MCPClientHandlers = MCPClientHandlers(), responseLimits: MCPResponseLimits = .default, + versionNegotiation: MCPVersionNegotiation = .auto, liveUpdates: Bool = true) +} + struct LocalLMLabState: Codable, Sendable, Equatable { static let currentVersion = 1 var version: Int @@ -2663,14 +2818,60 @@ struct LocalLMLabSession: Sendable { let route: RouteName let modelID: ModelID var retryOnContextOverflow: RetryPolicy // .disabled by default; set before respond() - var languageModelSession: LanguageModelSession { get } // Apple's session — use directly to opt out of the retry wrapper + var turnContext: (@Sendable () -> String?)? // sent ahead of every prompt (e.g. the date); not shown in hostTranscript + var hostTranscript: HostTranscript { get } // the conversation as a UI shows it (below) + var languageModelSession: LanguageModelSession { get } // Apple's session — the escape hatch; a turn run on it skips the SDK's turn handling var events: AsyncStream { get } var citations: [Citation] { get } // web-search sources accumulated this session ([§6b](#6b-online-providers--gpt-claude-online-openrouter-locallmlabsdkremote)) func reconcileServerToolArtifacts() // re-derive citations/events from the transcript after a manual edit var contextBudget: ContextBudget { get } func cancel() - // + respond(...) / streamResponse(...) wrappers (LocalLMLabSession+respond) that apply retryOnContextOverflow - func respond(to prompt: String, options: SessionOptions) async throws -> String // per-call options, on-device model only + // Running a turn: hostTranscript, turnContext, widget context, MCP tool refresh, retryOnContextOverflow + @discardableResult + func respond(to prompt: String, options: SessionOptions = .init(), // per-call options: on-device model only + fromAppInstance instance: String? = nil) async throws -> String // instance: a message an MCP App view sent + func streamResponse(to prompt: String, options: SessionOptions = .init(), + fromAppInstance instance: String? = nil) -> AsyncThrowingStream // the reply so far + // Outside the model's loop (MCP App views, your app): same authorizer, recorded in hostTranscript + @MainActor func callMCPTool(server: MCPServerID, tool: String, arguments: [String: MCPValue], + initiator: ToolCallInitiator) async -> Result + @MainActor func updateModelContext(_ text: String, fromAppInstance instance: String) // sent with the next turn +} + +@MainActor @Observable final class HostTranscript { + var entries: [Entry] { get } // in model order; append-only + var replyInProgress: ReplyInProgress? { get } // while streamResponse runs + var toolCalls: [ToolCallRecord] { get } + func toolCall(id: String) -> ToolCallRecord? + func appOnlyCallCount(for instance: String) -> Int // an MCP App view's calls to app-only tools (not recorded) + var resultMemoryBudget: Int // bytes of full MCP results kept in memory (8 MB); the rest spill to disk + var spillDirectory: URL? // nil: a temp directory the SDK deletes + func archive() throws -> Data // what the user saw, incl. full results + func restore(from data: Data) throws // into an empty transcript; ArchiveError otherwise + struct Entry: Identifiable, Sendable { + let id: String; var content: Content; let turn: Int + var appInstance: String? // set when an MCP App view sent it + enum Content: Sendable { // non-frozen: switch with @unknown default + case user(String), assistant(String), reasoning(String) // an inline block becomes .reasoning + case toolCall(ToolCallRecord) + case appContext(instance: String, text: String) + } + } + struct ReplyInProgress: Sendable, Equatable { var reasoning: String?; var text: String; var isReasoning: Bool } + enum ArchiveError: Error, Equatable { case notEmpty, unsupportedFormat(String) } +} +struct ToolCallRecord: Identifiable, Sendable { + let id: String // same as the SessionEvent id + var modelCallID: String? // the model's own call id, when the model made the call + let tool: ToolReference // .mcp(server:tool:) | .host(name:) | .provider(ServerToolActivity) + let initiator: ToolCallInitiator // .model | .app(instance:) | .host + let arguments: GeneratedContent?; let argumentsDescription: String + var state: State // .awaitingApproval | .running | .finished | .failed(reason:) | .denied(reason:) + var output: String? // what the model saw + var mcpResult: MCPToolResult? { get } // the full result (read back from disk if spilled) + let app: MCPAppLink? // the tool's MCP App view, if any + let toolDescriptor: MCPToolDescriptor? + let startedAt: Date; var finishedAt: Date? } enum SessionEvent: Sendable { case modelLoadProgress(fraction: Double) @@ -3273,6 +3474,15 @@ struct MCPToolDescriptor: Codable, Sendable { var enabled: Bool // defaults to false for newly-discovered tools var title: String? // human display name for UI (2025-06-18+); fall back to name var outputSchema: Data? // declared shape of structuredContent, if any (2025-06-18+) + var meta: MCPValue? // the tool's _meta + var annotations: MCPToolAnnotations? // readOnlyHint / destructiveHint …: the server's claim (believed only for a trusted server) + var app: MCPAppLink? { get } // its MCP App view (_meta.ui.resourceUri), if any + var isModelVisible: Bool { get } // false for app-only tools: never offered to a model +} +struct MCPAppLink: Sendable, Equatable, Hashable { var resourceURI: String; var visibility: Set } +enum MCPAppVisibility: String, Sendable, Codable, CaseIterable { case model, app } +struct MCPToolAnnotations: Codable, Sendable, Equatable { + var title: String?; var readOnlyHint: Bool?; var destructiveHint: Bool?; var idempotentHint: Bool?; var openWorldHint: Bool? } struct MCPResourceDescriptor: Codable, Sendable { @@ -3282,6 +3492,8 @@ struct MCPResourceDescriptor: Codable, Sendable { var description: String var mimeType: String? var enabled: Bool // defaults to true + var meta: MCPValue? + var isMCPAppResource: Bool { get } // a ui:// view's HTML: left out of model-facing resource lists } struct MCPResourceTemplateDescriptor: Codable, Sendable { @@ -3321,6 +3533,7 @@ struct MCPToolResult: Sendable, Codable, Equatable { var resourceLinks: [MCPResourceLink] var isError: Bool // the tool ran and reported failure var truncated: Bool // response hit MCPResponseLimits + var meta: MCPValue? // the result's _meta var renderedForModel: String { get } // compact, context-sized rendering folding in all of the above } struct MCPResourceLink: Sendable, Codable, Equatable { @@ -3350,8 +3563,12 @@ struct MCPClientHandlers: Sendable { init(elicitation: (any MCPElicitationHandler)? = nil, sampling: (any MCPSamplingHandler)? = nil, roots: (any MCPRootsProvider)? = nil, - logging: (any MCPLoggingSink)? = nil) + logging: (any MCPLoggingSink)? = nil, + extensions: [String: MCPValue] = [:]) // client capability extensions declared at initialize + var extensions: [String: MCPValue] } +// LocalLMLabSDKMCPAppsHost: MCPClientHandlers().advertisingMCPApps() — declares MCP Apps support +// (extensions["io.modelcontextprotocol/ui"]), which servers check before offering views. protocol MCPElicitationHandler: Sendable { func handleElicitation(_ request: MCPElicitationRequest) async -> MCPElicitationResponse @@ -3497,6 +3714,69 @@ boolean, string enums) convert cleanly. Anything past that — `oneOf`/`anyOf` u `const`, regex `pattern` — degrades to a free-form string leaf rather than failing the whole tool, since the remote server is still the real source of argument validation. +### MCP Apps (`LocalLMLabSDKMCPAppsHost`) + +Open source, shipped as source like `Components`, on Core's public API only. Walkthrough: +[§3f](#3f-mcp-apps-showing-a-servers-interactive-views). + +```swift +// --- declare support, fetch and check a view ---------------------------------- +extension MCPClientHandlers { func advertisingMCPApps() -> MCPClientHandlers; static let mcpAppsExtensionKey: String } +struct MCPAppResource: Sendable { // a validated view: text/html;profile=mcp-app, size-capped + static let mimeType = "text/html;profile=mcp-app" + let html: String; let sha256: String // pin / compare versions by sha256 + init(content: MCPResourceContent, requestedURI: String, listedMeta: MCPValue? = nil) throws +} +@MainActor final class MCPAppWidgetCache { // views on disk by hash; LRU past maxBytes (64 MB) + init(directory: URL? = nil, maxBytes: Int = 64 * 1024 * 1024) + func fetch(resourceURI: String, server: MCPServerID, manager: MCPServerManager, recordID: String? = nil) async throws -> MCPAppResource + func store(_ content: MCPResourceContent, requestedURI: String) throws -> MCPAppResource + func resource(sha256: String) -> MCPAppResource? // nil if missing or tampered + func bind(recordID: String, sha256: String); func sha256(forRecord recordID: String) -> String? +} + +// --- render --------------------------------------------------------------------- +@MainActor final class MCPAppViewController: NSObject, ObservableObject { + init(resource: MCPAppResource, tools: [MCPToolDescriptor], backend: any MCPAppsBackend, + configuration: MCPAppsBridgeConfiguration, + bridgePolicy: any MCPAppsBridgePolicy = MCPAppsDefaultPolicy(), // tool calls allowed; messages, context, links denied + sandboxPolicy: MCPAppsSandboxPolicy = .closed, // no network unless allowed + actions: MCPAppsHostActions = MCPAppsHostActions(), + supportedDisplayModes: [String] = ["inline"], + audit: @escaping @MainActor (MCPAppsAuditEvent) -> Void = { _ in }) + let webView: WKWebView; let bridge: MCPAppsBridge; let resource: MCPAppResource + var contentHeight: CGFloat? { get } // the view's reported height + func load() async; func close() async // after close(), release the controller to end its process +} +struct MCPAppView: NSViewRepresentable { init(controller: MCPAppViewController) } + +// --- a view's calls go through the session ---------------------------------------- +struct MCPAppsSessionBackend: MCPAppsBackend { // tools/call → session.callMCPTool(…, initiator: .app(instance:)) + init(session: LocalLMLabSession, manager: MCPServerManager, server: MCPServerID, instance: String) +} +extension MCPAppsHostActions { // ui/message → a turn; ui/update-model-context → model context + static func session(_ session: LocalLMLabSession, instance: String, openLink: … = { _ in }, + onReply: … = { _ in }, requestDisplayMode: … = { _ in false }) -> MCPAppsHostActions +} + +// --- many views in one conversation ------------------------------------------------- +@MainActor final class MCPAppViewPool: ObservableObject { + init(limit: Int = 3); var limit: Int; var liveKeys: [String] { get } + func controller(for key: String, make: () throws -> MCPAppViewController) rethrows -> MCPAppViewController + func touch(_ key: String); func isLive(_ key: String) -> Bool; func snapshot(for key: String) -> NSImage? + func release(_ key: String) async; func releaseAll() async +} +enum MCPAppRecreation { + static func revalidate(_ record: ToolCallRecord, source: any MCPAppWidgetSource, cache: MCPAppWidgetCache, + approvedSHA256: Set? = nil) async -> Revalidation? + static func deliver(_ record: ToolCallRecord, to controller: MCPAppViewController) -> Bool // stored arguments + result + struct Revalidation { var decision: MCPAppRecreationDecision; var resource: MCPAppResource?; var widgetChanged: Bool } +} +enum MCPAppRecreationDecision { case render(viewOnly: Bool), needsRefresh(mayRefreshAutomatically: Bool), notApproved, unavailable } +struct MCPServerManagerWidgetSource: MCPAppWidgetSource { init(manager: MCPServerManager, server: MCPServerID) } +// MCPResponseLimits.maxUIResourceBytes (Core) caps a view's HTML (4 MB). +``` + ### Components (`LocalLMLabSDKComponents`) ```swift From 890b6ce76ab831899f05e8bd69420821103f9082 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:52:23 -0700 Subject: [PATCH 06/73] aiql (synced): run the turn with session.respond, not languageModelSession 2.0 (docs/21 #7): a turn run on languageModelSession skips the context-overflow retry, the per-turn MCP tool refresh and the host transcript. session.events progress is unchanged. Verified live: AIQL's pipeline code, run headless against econ-index.mcp.claude.com with Qwen3-8B ("every country and its usage index, highest first, top 10"): the data tool saved 129 KB to raw/data.json via saveAs, loadTable + one sqlQuery wrote the right top-10 out.csv, and the file-backed setting was cleared after the run. Co-Authored-By: Claude --- examples/aiql/Sources/AIQL/AIQLApp.swift | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/examples/aiql/Sources/AIQL/AIQLApp.swift b/examples/aiql/Sources/AIQL/AIQLApp.swift index 1fc9bfe..e02291a 100644 --- a/examples/aiql/Sources/AIQL/AIQLApp.swift +++ b/examples/aiql/Sources/AIQL/AIQLApp.swift @@ -278,8 +278,10 @@ final class AIQLModel: ObservableObject { } defer { stepTask.cancel() } + // The session's own turn method, not `languageModelSession.respond`: that escape hatch + // skips the context-overflow retry, the MCP tool refresh and the host transcript. do { - _ = try await session.languageModelSession.respond(to: "Question: \(prompt)\n\nBegin with step 1 now.") + try await session.respond(to: "Question: \(prompt)\n\nBegin with step 1 now.") } catch { return .failed(await GenerationErrorDescription.describe(error)) } From 1f7cc96dc54eaaf7c919a1b05b060c293b5337c0 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:53:37 -0700 Subject: [PATCH 07/73] docs: MRTR lines now log under their own MCP.mrtr category --- CHANGELOG.md | 2 +- docs/mcp-diagnostics.md | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 02b7a26..f344b8a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -124,7 +124,7 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) `MCPListenEnd` (default implementation ends as refused). - `MCPProtocolVersion.supportsURLElicitation` and `.sendsProtocolVersionHeader` feature gates. - Diagnostics: the `server/discover` probe and fallback, live-update status, and MRTR rounds - (export area `mrtr`) are logged ([`docs/mcp-diagnostics.md`](docs/mcp-diagnostics.md)). + (new category `MCP.mrtr`) are logged ([`docs/mcp-diagnostics.md`](docs/mcp-diagnostics.md)). - Authorization fixes found by the official MCP conformance suite: protected-resource metadata at the path-suffixed then root well-known URL; `resource` must cover the server URL; metadata `issuer` and redirect `iss` are checked (RFC 8414, RFC 9207); a changed authorization server diff --git a/docs/mcp-diagnostics.md b/docs/mcp-diagnostics.md index 4925f71..8423781 100644 --- a/docs/mcp-diagnostics.md +++ b/docs/mcp-diagnostics.md @@ -47,7 +47,7 @@ defaults write MCPLogLevel debug # then relaunch ## Layer 1: reading Apple's unified log - **subsystem:** `ai.thisbrain.locallmlab.sdkcore` -- **categories:** `MCP.connection`, `MCP.manager`, `MCP.oauth`, `MCP.stream` +- **categories:** `MCP.connection`, `MCP.manager`, `MCP.oauth`, `MCP.stream`, `MCP.mrtr` ```bash # live: @@ -113,7 +113,7 @@ a glance before sending it on. |---|---| | `MCP.connection` | the `server/discover` probe and its outcome — negotiated stateless, rejected and falling back to `initialize`, no common version, or a sign-in failure (which does not fall back); the protocol version offered and agreed, the server's name and capabilities; a server on the unsupported `2024-11-05` HTTP+SSE transport; a stateless request retried at another version after "unsupported protocol version"; a `subscriptions/listen` stream opening and ending (with the reason); every method call (at `debug`); any non-2xx HTTP response with its content type; a 401 and the start of sign-in (with the `WWW-Authenticate` header); a 403 `insufficient_scope` and the scope step-up; a session id being issued; a personal access token being rejected | | `MCP.manager` | `addServer` and its result (agreed version, tool count); live updates becoming active (what the server granted), unavailable (why), or resuming after sign-in; a live update re-listing tools; a tool result whose structured data didn't match the tool's declared output schema | -| `MCP.mrtr` *(export only)* | `2026-07-28` multi-round-trip requests: a call coming back `input_required` (which inputs it asked for); an input the client can't answer (no handler registered, or elicitation suppressed for that call); a call stopped for exceeding the 8-round limit. These lines are labelled `MCP.mrtr` in `exportText()` / `exportJSON()`, but in Console and `log stream` they appear under the `MCP.connection` category. | +| `MCP.mrtr` | `2026-07-28` multi-round-trip requests: a call coming back `input_required` (which inputs it asked for); an input the client can't answer (no handler registered, or elicitation suppressed for that call); a call stopped for exceeding the 8-round limit | | `MCP.oauth` | the protected-resource metadata fetch; which `.well-known` discovery URL resolved (or that none did); the choice between Dynamic Client Registration, CIMD, and a manual client id — and why CIMD was skipped if it was; the PKCE check; the browser being opened (endpoint, `client_id`, scope); an `iss` mismatch; the code coming back; the token-exchange HTTP status; whether a silent refresh worked; every failure with the server's actual (truncated, redacted) response | | `MCP.stream` | each server-sent-event frame as it's classified — an inbound request (`elicitation/create` and the like), a notification, a result; a server error result; a stream that broke mid-request; a stream that ended without the result it was waiting for | From 4402faf070074184ffaa4362bc2ac4497afbbcde Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:54:11 -0700 Subject: [PATCH 08/73] =?UTF-8?q?docs:=20SDK=20guide=20=E2=80=94=20per-ser?= =?UTF-8?q?ver=20trust=20and=20approval,=20one=20authorizer=20(2.0)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §7c lever 2 rewritten for 2.0: ConfirmingToolAuthorizer(channel:policy:hostTools:timeout:), per-server trust / approval on lab.mcp (settings table), ToolPolicy, fail-closed rules, live reads, persistence after restore, cross-process approvals. §12: PendingToolCall / Summary with initiator + serverApproval, ToolCallInitiator, MCPServerTrust, ToolApproval, ToolApprover, ToolPolicy, ToolPolicyDecision; RuleBasedToolAuthorizer and Requirement removed; MCPServerManager setTrust / setToolApproval / toolApproval(server:); MCPServerState trust fields. §11 / Components: the picker's trust and approval row and notices. Co-Authored-By: Claude --- docs/sdk-guide.md | 165 +++++++++++++++++++++++++++++++--------------- 1 file changed, 112 insertions(+), 53 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 676ec28..ec045a9 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1918,49 +1918,87 @@ declares its impact (`ImpactRatedTool`). is the checkpoint between "the model chose to call this" and "the side effect happens." It's opt-in: with no `authorizer:`, `makeSession` installs no invocation gate — every tool in the list is callable by the model, with whatever arguments it chooses, and every call runs -immediately with no confirmation and no way to deny it. That is the pre-authorizer behaviour, -unchanged. +immediately with no confirmation and no way to deny it. + +The SDK's one authorizer is **`ConfirmingToolAuthorizer`**. You say **who may approve what** +where the tools come from — per MCP server on `lab.mcp`, and for your own tools on the +authorizer — and it applies that on every call (new in 2.0): ```swift +// Per MCP server, on lab.mcp. The server is the unit of trust. +lab.mcp.setTrust(.trusted, server: companyServer) // believe its read-only / destructive labels +lab.mcp.setToolApproval(.allow, server: companyServer) // its tools run without asking +// An untrusted server (the default) asks the user before every call. + let session = try lab.makeSession( route: "chat", - tools: tools, - authorizer: RuleBasedToolAuthorizer( - rules: [.confirm(atOrAbove: .mutate), .denyMCPTools], - confirm: { call in await myConfirmationUI.ask(call) })) + tools: tools, // your own tools + authorizer: ConfirmingToolAuthorizer( + channel: presenter, // how the user is asked + policy: { call in await myPolicyService.decide(call) }, // optional external function + hostTools: .ask(atOrAbove: .mutate, by: .user))) // your own tools' approval (the default) ``` -- **`RuleBasedToolAuthorizer`** (Core) is pure logic: `denyTool` / `denyMCPTools` / - `deny(atOrAbove:)` / `allowTool` / `confirm(atOrAbove:)` / `confirmMCPTools`, - most-restrictive-wins. A `.confirm…` rule invokes your `confirm` closure; with no closure it - fails closed to deny. -- **`ConfirmingToolAuthorizer`** (Core) is "ask a human per call." It takes a - `ToolConfirmationChannel`; in a single-process SwiftUI app that's - `ToolConfirmationPresenter` (Components) plus `.toolConfirmationSheet(presenter)` on a root - view — no sheet UI of your own. `authorize(_:)` is `async`, and this denies after a timeout - (default 120s) so a forgotten sheet can't pin a turn open. +**The settings at a glance** (per server in 2.0; a tool has only on/off for now): + +| Level | Setting | Options | What it decides | +|---|---|---|---| +| Server | **Trust** — `setTrust(_:server:)` | `.untrusted` (default) · `.trusted` | Whether the server's own labels (`readOnlyHint`, `destructiveHint`) are believed. Untrusted: every tool counts as one that may change data (`.mutate`). | +| Server | **Approval** — `setToolApproval(_:server:)` | `.allow` — nothing asks
`.ask(atOrAbove: .mutate, …)` — reads run, changes and deletions ask
`.ask(atOrAbove: .destructive, …)` — only deletions ask
`.ask(atOrAbove: .read, …)` — every call asks
`nil` — the default for the trust level: untrusted asks before changes, trusted before deletions | Which calls need authorization before they run. | +| Server | ↳ **who answers** an ask — the `by:` of `.ask` | `.user` — the confirmation channel · `.policy` — your `ToolPolicy` (allow / deny / ask the user) | Who authorizes the calls that need it. | +| Tool | **On / Off** — `setToolEnabled(server:tool:enabled:)` | on · off (new tools start off) | Whether the model is offered the tool at all. | +| Tool | **Approval** — *a later release* | its own `.allow` / `.ask`, overriding the server's | e.g. allow `find-tasks` on an untrusted server, or always ask before `delete-object` on a trusted one. | + +- **Trust is only ever yours or the user's to give** — never anything the server says. + `MCPServerState.toolsChangedSinceTrusted` turns true when a trusted server's tools change, so + the user can review and re-confirm (`setTrust(.trusted, …)` again); trust isn't dropped + automatically. +- **The policy** (`ToolPolicy`) is your external authorization function — a policy service, an + IT rule set, a rule over the arguments. It returns `.allow`, `.deny(reason:)` or `.askUser`. + Use it for anything the per-server settings can't say (a per-tool allow list, a + per-connector rule). +- **Fails closed.** No channel when the user must answer (a headless host: pass + `channel: nil`), no policy when the policy must, a timeout (default 120s), or a declined + question all deny. +- **Read live.** Settings are read on every call, so a change applies to a running session's + next call — the model's and an MCP App view's alike. A view's answer is remembered per view + and tool, so a polling dashboard asks once; deletions are asked each time. +- **Not persisted by the manager.** Save `MCPServerState.trust` / `.toolApproval` with your + server list and set them again after `restore(from:)` (before or after both work). +- **To stop being asked about deletions** on a server you rely on, set `.allow`, or route them + to your policy with `.ask(atOrAbove: .destructive, by: .policy)` so a rule decides and no one + is interrupted. +- In a single-process SwiftUI app the channel is `ToolConfirmationPresenter` (Components) plus + `.toolConfirmationSheet(presenter)` on a root view — no sheet UI of your own. Components' + `MCPServerPickerView` shows each server's trust and approval. - A denied call comes back to the model as the tool result `"DENIED: "` (for `String`-returning tools — all the SDK's and `MCPTool`), so the model adapts rather than the - turn failing. + turn failing. A policy denial reads `"denied by policy: "`. The authorizer is a floor, not a ceiling: it can't grant access to a tool that isn't in the list, and it doesn't restrain your own code calling `CalendarAccess.deleteEvent(...)` directly. -It gates model-initiated calls only. +It gates model- and view-initiated calls only. **Across processes.** If your app runs the session in a headless helper and the UI in another process, implement `ToolConfirmationChannel` with your transport — that's the *only* security code you write. `PendingToolCall.summary` is a `Codable` `PendingToolCallSummary` (`toolName` / -`argumentsDescription` / `origin` / `impact`; drops the non-serializable `arguments`) built for -exactly that hop; `DecisionGate` (Core) handles the resume-once race between the reply and a -cancellation. - -**The SDK can't see inside an MCP tool.** It didn't write it, so it can't know if a call is a -read or a delete — every `MCPTool` is rated `.mutate`. `limited(toMaxImpact:)` can only include -or exclude a whole MCP server, not grade it, and `ConfirmingToolAuthorizer` will confirm even a -read-shaped MCP call. With `makeSession(includeMCPTools: -true)` the SDK tags them `.mcp` origin so `denyMCPTools` / `confirmMCPTools` and a rule's -`call.origin` check apply. A host that builds its own MCP-backed `Tool` type (e.g. one proxying -to a connection in another process) conforms it to `OriginTaggedTool` to keep that origin. +`argumentsDescription` / `origin` / `impact` / `initiator` / `serverApproval`; drops the +non-serializable `arguments`) built for exactly that hop; `DecisionGate` (Core) handles the +resume-once race between the reply and a cancellation. If the MCP connections live in the other +process, record each server's approval on the helper's `lab.mcp` before `makeSession` — +approvals are keyed by server id and need no connection. + +**What the SDK can see inside an MCP tool.** It didn't write the tool, so on an untrusted server +every call is `.mutate`: `limited(toMaxImpact:)` can only include or exclude a whole server, +and the default approval asks even before a read-shaped call. Trust the server and its own +`readOnlyHint` / `destructiveHint` set each tool's impact. With `makeSession(includeMCPTools: +true)` the SDK tags the tools with their server (`.mcp` origin), which is how the server's +approval applies. A host that builds its own MCP-backed `Tool` type (e.g. one proxying to a +connection in another process) conforms it to `OriginTaggedTool` to keep that origin. + +**Coming from 1.x.** `RuleBasedToolAuthorizer`, `ConfirmingToolAuthorizer`'s `requirement:` +closure and `setTrustsToolAnnotations` are gone — see +[migrating-to-2.0.md](migrating-to-2.0.md) §3 for the mapping. ## 8. Filesystem access: security-scoped bookmarks (example, not in Core) @@ -2368,8 +2406,12 @@ dependency on `Core.xcframework` and no source access to Core's internals. SwiftUI apps that want `@Published`-style reactivity without writing the wrapper themselves (see [§6](#6-general-api-reference)'s note). - **`MCPServerPickerView`** — add/list/reconnect/disconnect/remove MCP servers, all three auth - types from [§3](#3-connecting-to-an-mcp-server-three-auth-options-and-how-to-pick-between-them), per-tool and per-resource enable/disable, and a "Save As…" action that - exports a server's tools/resources/prompts to a text file. + types from [§3](#3-connecting-to-an-mcp-server-three-auth-options-and-how-to-pick-between-them) (picked with a radio group), per-tool and per-resource enable/disable, and a "Save As…" action that + exports a server's tools/resources/prompts to a text file. New in 2.0: each server row has its + **Trust** and **"Before a tool runs"** approval ([§7c](#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first)) — an approval your app routes to its + policy shows as "Decided by the app's policy" and isn't editable there — plus a notice when none + of a server's tools are on (so a model can't use it yet) and one when a trusted server's tools + changed since it was trusted ("Trust Again"). - **`MCPOAuthWaitingView`** — shown while an OAuth sign-in is in flight in the system browser; `MCPServerPickerView` already uses this internally during its own add-server flow. - **`MCPResourcesView`** / **`MCPPromptsView`** — browse a session's enabled resources/prompts and @@ -3011,39 +3053,45 @@ struct PendingToolCall: Sendable { let arguments: GeneratedContent? // structured; nil only for a rare third-party From-only Arguments let argumentsDescription: String // always present let origin: ToolOrigin // .host | .mcp(server:displayName:) - let impact: ToolImpact - init(toolName: String, arguments: GeneratedContent?, argumentsDescription: String, origin: ToolOrigin, impact: ToolImpact) + let impact: ToolImpact // MCP: .mutate unless the server is trusted (then its annotations) + let initiator: ToolCallInitiator // .model | .app(instance:) (an MCP App view) | .host + let serverApproval: ToolApproval? // MCP: the server's approval at call time, filled by the SDK; nil for your own tools + init(toolName: String, arguments: GeneratedContent?, argumentsDescription: String, origin: ToolOrigin, impact: ToolImpact, + initiator: ToolCallInitiator = .model, serverApproval: ToolApproval? = nil) var summary: PendingToolCallSummary { get } // the Codable projection, for crossing a process boundary } -struct PendingToolCallSummary: Codable, Sendable, Equatable { +struct PendingToolCallSummary: Codable, Sendable, Equatable { // a missing initiator decodes as .model, serverApproval as nil let toolName: String; let argumentsDescription: String; let origin: ToolOrigin; let impact: ToolImpact + let initiator: ToolCallInitiator; let serverApproval: ToolApproval? + init(toolName: String, argumentsDescription: String, origin: ToolOrigin, impact: ToolImpact, + initiator: ToolCallInitiator = .model, serverApproval: ToolApproval? = nil) } enum ToolOrigin: Sendable, Codable, Equatable { case host; case mcp(server: MCPServerID, displayName: String) } +enum ToolCallInitiator: Sendable, Hashable, Codable { case model; case app(instance: String); case host } -// makeSession(..., authorizer:) — [§12](#12-full-functiontype-reference) "The model layer" — installs the gate; nil = unchanged, no gate. - -struct RuleBasedToolAuthorizer: ToolCallAuthorizer { // pure logic, most-restrictive rule wins - enum Rule: Sendable { - case denyTool(String) - case denyMCPTools - case deny(atOrAbove: ToolImpact) - case allowTool(String) - case confirm(atOrAbove: ToolImpact) - case confirmMCPTools - } - init(rules: [Rule], denyReason: String = "blocked by this app's tool policy", - confirm: @escaping @Sendable (PendingToolCall) async -> Bool = { _ in false }) - static func confirmingMutations(confirm: @escaping @Sendable (PendingToolCall) async -> Bool) -> RuleBasedToolAuthorizer - static func confirmingDestructive(confirm: @escaping @Sendable (PendingToolCall) async -> Bool) -> RuleBasedToolAuthorizer +// --- per-server trust and approval (new in 2.0; set on lab.mcp — see "MCP client" below) --- +enum MCPServerTrust: String, Sendable, Codable, CaseIterable { case untrusted, trusted } // trusted ⇒ annotations set impact +enum ToolApprover: String, Sendable, Codable { case user, policy } +enum ToolApproval: Sendable, Codable, Equatable { + case allow // every call runs + case ask(atOrAbove: ToolImpact, by: ToolApprover) // calls at/above the impact need authorization by `by` + static func `default`(for trust: MCPServerTrust) -> ToolApproval // untrusted: .ask(.mutate, .user); trusted: .ask(.destructive, .user) } +typealias ToolPolicy = @Sendable (PendingToolCall) async -> ToolPolicyDecision // your external authorization function +enum ToolPolicyDecision: Sendable, Equatable { case allow; case deny(reason: String); case askUser } + +// makeSession(..., authorizer:) — [§12](#12-full-functiontype-reference) "The model layer" — installs the gate; nil = unchanged, no gate. -protocol ToolConfirmationChannel: Sendable { // the transport a "ask a human" authorizer suspends on +protocol ToolConfirmationChannel: Sendable { // the transport an "ask the user" decision suspends on func requestDecision(for call: PendingToolCall) async -> Bool } -struct ConfirmingToolAuthorizer: ToolCallAuthorizer { // "ask a human per call," via any ToolConfirmationChannel - enum Requirement: Sendable { case allow; case confirm; case deny(reason: String) } - init(channel: any ToolConfirmationChannel, timeout: Duration = .seconds(120), - requirement: @escaping @Sendable (PendingToolCall) -> Requirement = { _ in .confirm }) +struct ConfirmingToolAuthorizer: ToolCallAuthorizer { // the SDK's one authorizer + // Per call: the MCP server's approval (call.serverApproval; nil ⇒ the untrusted default) or `hostTools` + // for your own tools; below the threshold runs, .ask(by: .user) asks `channel`, .ask(by: .policy) asks + // `policy` (which may hand it to the user). Fails closed: no channel / no policy / timeout / declined ⇒ deny. + // A view's (initiator .app) answer is remembered per view, server and tool; .destructive and timeouts aren't. + init(channel: (any ToolConfirmationChannel)?, policy: ToolPolicy? = nil, + hostTools: ToolApproval = .ask(atOrAbove: .mutate, by: .user), timeout: Duration = .seconds(120)) } final class DecisionGate: @unchecked Sendable { // guards the resume-once race between a reply and the timeout init() @@ -3388,6 +3436,11 @@ final class MCPServerManager { func toolsForSession() -> [MCPToolDescriptor] func setToolEnabled(server: MCPServerID, tool: String, enabled: Bool) + // Per-server trust and approval ([§7c](#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first), new in 2.0). + // Kept across reconnects, cleared by removeServer; not persisted — save the state's trust / toolApproval and set them again after restore. + func setTrust(_ trust: MCPServerTrust, server: MCPServerID) // .trusted ⇒ its annotations set each tool's impact + func setToolApproval(_ approval: ToolApproval?, server: MCPServerID) // nil ⇒ ToolApproval.default(for: trust) + func toolApproval(server: MCPServerID) -> ToolApproval // the effective approval, read on every call // -> MCPToolResult ([§3b](#3b-what-a-tool-call-gives-you-back-mcptoolresult)), not a String. A tool that ran but failed is .success with isError == true. // allowElicitation: false on a code path that can't show a prompt (headless / background). func callTool(server: MCPServerID, tool: String, arguments: [String: MCPValue], allowElicitation: Bool = true) async -> Result @@ -3423,6 +3476,10 @@ struct MCPServerState: Codable, Sendable { var negotiatedProtocolVersion: MCPProtocolVersion? // nil until connected ([§3a](#3a-which-mcp-revision-the-client-speaks--and-why-you-mostly-dont-have-to-care)) var serverInstructions: String? // server's own usage note (initialize or server/discover) var liveUpdates: MCPLiveUpdates? // stateless servers only; nil otherwise ([§3a](#3a-which-mcp-revision-the-client-speaks--and-why-you-mostly-dont-have-to-care)) + var trust: MCPServerTrust // setTrust (default .untrusted) + var toolApproval: ToolApproval? // setToolApproval's value; nil = default for the trust level + var trustedToolsDigest: String? // the tool list when last trusted + var toolsChangedSinceTrusted: Bool { get } // a trusted, connected server whose tools changed — show it for review func exportSummary() -> String // plain-text listing of tools/resources/prompts, enabled state, token cost } @@ -3793,6 +3850,8 @@ struct MCPServerPickerView: View { init(manager: MCPServerManagerObservable) // Add/list/reconnect/disconnect/remove, all three auth types, per-tool and per-resource // enable/disable, prompts listing (read-only), and a "Save As…" text export per server. + // Per server (2.0): trust + approval controls (setTrust / setToolApproval), a "no tools on" notice, + // and a "tools changed since trusted" notice with Trust Again. } struct MCPOAuthWaitingView: View { From cd638190844293c8dbd819ae297ce1de0c4ad270 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 17:56:44 -0700 Subject: [PATCH 09/73] security-demo (synced) + walkthrough: 2.0 per-server approval MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Synced the example's sources and README from the SDK repo: Todoist's approval is set on lab.mcp (setToolApproval: nil = ask, .allow = don't), Calendar's is the authorizer's hostTools, and the authorizer is ConfirmingToolAuthorizer(channel:hostTools:) — the 1.x requirement closure is gone. annotated-examples.md re-quotes DemoPolicy and run() from the migrated code and updates the tally. Package.swift is unchanged (still defaults to the 1.0.0-GA binaries, like Components/Package.swift). Co-Authored-By: Claude --- docs/annotated-examples.md | 40 +++++++++++-------- examples/security-demo/README.md | 19 ++++++--- .../Sources/SecurityDemo/AppModel.swift | 8 ++-- .../Sources/SecurityDemo/ContentView.swift | 2 +- .../Sources/SecurityDemo/DemoSecurity.swift | 22 +++++----- 5 files changed, 51 insertions(+), 40 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index b2e2be6..68ded01 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -2226,7 +2226,9 @@ seam that keeps `Components` free of any dependency on `Remote`. ### `Sources/SecurityDemo/DemoSecurity.swift` -This file is the whole idea: a "Security panel" is **two SDK levers**, and nothing else. `DemoSecurity` is the observable UI state; `DemoPolicy` is the immutable snapshot a run takes so editing the panel mid-turn can't change a session already built (the same split `SecurityPolicy` uses in LocalLM Lab). +This file is the whole idea: a "Security panel" is **two SDK levers**, and nothing else. In 2.0 +the second lever is stated **where the tools come from** — the app's own tools on the +authorizer, an MCP server's on `lab.mcp` ([sdk-guide.md §7c](sdk-guide.md#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first)). `DemoSecurity` is the observable UI state; `DemoPolicy` is the immutable snapshot a run takes so editing the panel mid-turn can't change a session already built (the same split `SecurityPolicy` uses in LocalLM Lab). ```swift import FoundationModels @@ -2273,22 +2275,23 @@ struct DemoPolicy: Sendable { tools.limited(toMaxImpact: calendarLevel.maxImpact) // ← SDK — Sequence extension } - // Lever 2 — invocation. Per-call: reads always run; a mutating/destructive call is - // confirmed when its connector's toggle is on. - func requirement(for call: PendingToolCall) -> ConfirmingToolAuthorizer.Requirement { // ← SDK (types) - guard call.impact >= .mutate else { return .allow } // ← SDK — call.impact - switch call.origin { // ← SDK — .host vs .mcp - case .host: return calendarConfirm ? .confirm : .allow - case .mcp: return todoistConfirm ? .confirm : .allow - @unknown default: return .confirm - } + // Lever 2 — invocation, for the app's own tools (here only Calendar's): ask the user before + // a change when the toggle is on. Reads always run. + var calendarApproval: ToolApproval { // ← SDK — .allow | .ask(atOrAbove:by:) + calendarConfirm ? .ask(atOrAbove: .mutate, by: .user) : .allow // ← SDK + } + + // Lever 2 — invocation, per MCP server (set on lab.mcp). Todoist stays untrusted, so its tools + // all count as changes: nil (the untrusted default) asks before each call; .allow never asks. + var todoistApproval: ToolApproval? { // ← SDK + todoistConfirm ? nil : .allow } } ``` ### `Sources/SecurityDemo/AppModel.swift` -*130 lines of code (226 with comments).* `bootstrap()` is the host-app setup a real app would give proper UI (register providers, grant Calendar, connect Todoist MCP). `run()` is the payoff — `DemoPolicy` → a tool list + an authorizer → `lab.makeSession`. +*About 130 lines of code (230 with comments).* `bootstrap()` is the host-app setup a real app would give proper UI (register providers, grant Calendar, connect Todoist MCP). `run()` is the payoff — `DemoPolicy` → a tool list + an authorizer → `lab.makeSession`. ```swift import LocalLMLabSDKComponents // ← Components @@ -2354,10 +2357,12 @@ final class AppModel { UpdateCalendarEventTool(), DeleteCalendarEventTool(), ]) - // Lever 2 — invocation. nil when the panel asks for no confirmation. + // Lever 2 — invocation. Todoist's approval lives on its server in lab.mcp; the Calendar + // tools' on the authorizer. nil when the panel asks for no confirmation. + lab.mcp.setToolApproval(policy.todoistApproval, // ← SDK — per-server approval + server: MCPServerID(rawValue: todoistURL.absoluteString)) let authorizer: (any ToolCallAuthorizer)? = policy.wantsConfirmation // ← SDK (type) - ? ConfirmingToolAuthorizer(channel: presenter, // ← SDK - requirement: { call in policy.requirement(for: call) }) + ? ConfirmingToolAuthorizer(channel: presenter, hostTools: policy.calendarApproval) // ← SDK : nil let session = try lab.makeSession( // ← SDK @@ -2379,12 +2384,13 @@ final class AppModel { } ``` -**Tally**: of the file's 130 non-comment/non-blank lines, 21 touch the SDK directly (marked +**Tally**: of the file's ~130 non-comment/non-blank lines, 22 touch the SDK directly (marked above, plus 2 more marked `← Components`) — and every SDK one is either setup (`LocalLMLab()`, `registerProvider`, `addServer` + `setToolEnabled`, `CalendarAccess.requestAccess`) or the one `run()` call site: `route` → build `hostTools` with -`limited(toMaxImpact:)` → wrap in `ConfirmingToolAuthorizer` → `makeSession(authorizer:)` → -`respond`. `DemoSecurity.swift` adds 6 more, all in `DemoPolicy` — the two levers themselves. +`limited(toMaxImpact:)` → set Todoist's approval with `setToolApproval` → a +`ConfirmingToolAuthorizer` carrying Calendar's approval → `makeSession(authorizer:)` → +`respond`. `DemoSecurity.swift` adds 4 more, all in `DemoPolicy` — the two levers themselves. The `ToolConfirmationPresenter` + `.toolConfirmationSheet(_:)` (in `ContentView`, one line) is the entire confirmation UI. Nothing in the three panel views touches the SDK — they bind to `DemoSecurity`, and the snapshot does the rest. diff --git a/examples/security-demo/README.md b/examples/security-demo/README.md index 970dd70..b11e971 100644 --- a/examples/security-demo/README.md +++ b/examples/security-demo/README.md @@ -86,19 +86,24 @@ gated. ### Gate 2 — invocation (`ConfirmingToolAuthorizer`) -For tools that *are* in the session, "Confirm each" decides whether a call runs or asks first: +For tools that *are* in the session, "Confirm each" decides whether a call runs or asks first. +The approval is set where the tools come from — per MCP server on `lab.mcp`, and for the app's +own tools on the authorizer (docs/sdk-authority-model.md §9): ```swift +// Todoist (an MCP server): nil = the untrusted default, ask before each call; .allow = don't ask. +lab.mcp.setToolApproval(policy.todoistConfirm ? nil : .allow, server: todoist) + let authorizer = ConfirmingToolAuthorizer( channel: presenter, // Components.ToolConfirmationPresenter - requirement: { call in policy.requirement(for: call) }) + hostTools: policy.calendarConfirm ? .ask(atOrAbove: .mutate, by: .user) : .allow) let session = try lab.makeSession(route: "frontier", tools: hostTools, includeMCPTools: true, authorizer: authorizer) ``` -`requirement(for:)` returns `.allow` / `.confirm` / `.deny` per call. Here: reads always run; -a `.mutate`/`.destructive` call is confirmed when its connector's toggle is on. Deny a card and +Here: reads always run; a `.mutate`/`.destructive` call is confirmed when its connector's toggle +is on. Deny a card and the model gets `DENIED: not approved` back and continues — the turn doesn't crash. No answer within ~120s auto-denies (a forgotten sheet can't pin a turn open). @@ -115,7 +120,9 @@ confirm toggle — and even a search like `find-tasks` asks. LocalLM Lab's "don' session / always)" is how you'd quiet the safe ones; this demo leaves it noisy on purpose. Because the connection lives in this process, `makeSession(includeMCPTools: true)` builds the -SDK's own `MCPTool` and tags it `.mcp` — so `requirement(for:)` can apply MCP policy to it. +SDK's own `MCPTool` and tags it `.mcp` — so the server's approval on `lab.mcp` applies to it. +(Marking the server **trusted** — `lab.mcp.setTrust(.trusted, server:)` — would let its own +read-only / destructive labels split the tools; the demo keeps it untrusted.) --- @@ -124,7 +131,7 @@ SDK's own `MCPTool` and tags it `.mcp` — so `requirement(for:)` can apply MCP | Panel control | SDK | Where | |---|---|---| | Calendar **level** | `Sequence.limited(toMaxImpact:)` | [`DemoSecurity.swift`](Sources/SecurityDemo/DemoSecurity.swift) | -| **Confirm each** | `ConfirmingToolAuthorizer(channel:requirement:)` → `makeSession(authorizer:)` | [`AppModel.run()`](Sources/SecurityDemo/AppModel.swift) | +| **Confirm each** | Calendar: `ConfirmingToolAuthorizer(channel:hostTools:)`; Todoist: `lab.mcp.setToolApproval(_:server:)` → `makeSession(authorizer:)` | [`AppModel.run()`](Sources/SecurityDemo/AppModel.swift) | | the confirmation sheet | `Components.ToolConfirmationPresenter` + `.toolConfirmationSheet(_:)` | [`ContentView.swift`](Sources/SecurityDemo/ContentView.swift) — one line | `DemoSecurity` is `@MainActor @Observable` UI state; `DemoPolicy` is an immutable `Sendable` diff --git a/examples/security-demo/Sources/SecurityDemo/AppModel.swift b/examples/security-demo/Sources/SecurityDemo/AppModel.swift index e39b949..7a4a8ef 100644 --- a/examples/security-demo/Sources/SecurityDemo/AppModel.swift +++ b/examples/security-demo/Sources/SecurityDemo/AppModel.swift @@ -191,11 +191,11 @@ final class AppModel { DeleteCalendarEventTool(), ]) - // Lever 2 — invocation. No authorizer at all when nothing is set to confirm. + // Lever 2 — invocation. Todoist's approval lives on its server in `lab.mcp`; the + // Calendar tools' on the authorizer. No authorizer at all when nothing is set to confirm. + lab.mcp.setToolApproval(policy.todoistApproval, server: MCPServerID(rawValue: todoistURL.absoluteString)) let authorizer: (any ToolCallAuthorizer)? = policy.wantsConfirmation - ? ConfirmingToolAuthorizer( - channel: presenter, - requirement: { call in policy.requirement(for: call) }) + ? ConfirmingToolAuthorizer(channel: presenter, hostTools: policy.calendarApproval) : nil let session = try lab.makeSession( diff --git a/examples/security-demo/Sources/SecurityDemo/ContentView.swift b/examples/security-demo/Sources/SecurityDemo/ContentView.swift index a42f38d..eb84bf7 100644 --- a/examples/security-demo/Sources/SecurityDemo/ContentView.swift +++ b/examples/security-demo/Sources/SecurityDemo/ContentView.swift @@ -12,7 +12,7 @@ struct ContentView: View { .frame(minWidth: 420) } // The Components-provided sheet — one line, no sheet UI of our own. This is what - // `ConfirmingToolAuthorizer` drives every time `requirement(for:)` returns `.confirm`. + // `ConfirmingToolAuthorizer` drives every time a call's approval says to ask the user. .toolConfirmationSheet(model.presenter) } } diff --git a/examples/security-demo/Sources/SecurityDemo/DemoSecurity.swift b/examples/security-demo/Sources/SecurityDemo/DemoSecurity.swift index 3d2f166..c3eacea 100644 --- a/examples/security-demo/Sources/SecurityDemo/DemoSecurity.swift +++ b/examples/security-demo/Sources/SecurityDemo/DemoSecurity.swift @@ -63,17 +63,15 @@ struct DemoPolicy: Sendable { tools.limited(toMaxImpact: calendarLevel.maxImpact) } - /// Lever 2 — invocation: the per-call policy for `ConfirmingToolAuthorizer`. Reads always - /// run; a mutating/destructive call is confirmed when its connector's toggle is on. - func requirement(for call: PendingToolCall) -> ConfirmingToolAuthorizer.Requirement { - guard call.impact >= .mutate else { return .allow } - switch call.origin { - case .host: // the only host tools here are Calendar's - return calendarConfirm ? .confirm : .allow - case .mcp: - return todoistConfirm ? .confirm : .allow - @unknown default: - return .confirm - } + /// Lever 2 — invocation. The app's own tools (here only Calendar's): ask before a change when + /// the toggle is on. Reads always run. + var calendarApproval: ToolApproval { + calendarConfirm ? .ask(atOrAbove: .mutate, by: .user) : .allow + } + + /// Lever 2 — invocation, per MCP server (`lab.mcp.setToolApproval`). Todoist stays untrusted, so + /// its tools all count as changes: `nil` (the untrusted default) asks before each call. + var todoistApproval: ToolApproval? { + todoistConfirm ? nil : .allow } } From 419288dd896ea6988f1805c42570199d6c2254e7 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:03:05 -0700 Subject: [PATCH 10/73] docs: MRTR asks the user once per Cancel/Decline --- CHANGELOG.md | 5 ++++- docs/mcp-diagnostics.md | 2 +- docs/sdk-guide.md | 5 ++++- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f344b8a..a7c2f11 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -115,7 +115,10 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) `MCPSettings(versionNegotiation:)` or `MCPServerManager(versionNegotiation:)`. - **Multi-round-trip requests (MRTR).** A request that comes back `input_required` is answered through the existing `MCPClientHandlers` (elicitation in form or URL mode, sampling, roots) and - re-sent with the answers, for up to 8 rounds. No handler API changes. + re-sent with the answers, for up to 8 rounds. No handler API changes. A user's Cancel ends the + call without re-sending; after a Decline, a server that asks again is declined without + prompting the user a second time. Both come back as a `tools/call` tool error + (`isError`) the model can read, not a transport failure. - **Live updates** (`subscriptions/listen`). One stream per stateless server that announces list changes; tools, prompts and resources are re-listed on change with enabled flags kept. `MCPServerState.liveUpdates: MCPLiveUpdates?` (`.active` / `.reconnecting` / `.unavailable`). diff --git a/docs/mcp-diagnostics.md b/docs/mcp-diagnostics.md index 8423781..5244ee8 100644 --- a/docs/mcp-diagnostics.md +++ b/docs/mcp-diagnostics.md @@ -113,7 +113,7 @@ a glance before sending it on. |---|---| | `MCP.connection` | the `server/discover` probe and its outcome — negotiated stateless, rejected and falling back to `initialize`, no common version, or a sign-in failure (which does not fall back); the protocol version offered and agreed, the server's name and capabilities; a server on the unsupported `2024-11-05` HTTP+SSE transport; a stateless request retried at another version after "unsupported protocol version"; a `subscriptions/listen` stream opening and ending (with the reason); every method call (at `debug`); any non-2xx HTTP response with its content type; a 401 and the start of sign-in (with the `WWW-Authenticate` header); a 403 `insufficient_scope` and the scope step-up; a session id being issued; a personal access token being rejected | | `MCP.manager` | `addServer` and its result (agreed version, tool count); live updates becoming active (what the server granted), unavailable (why), or resuming after sign-in; a live update re-listing tools; a tool result whose structured data didn't match the tool's declared output schema | -| `MCP.mrtr` | `2026-07-28` multi-round-trip requests: a call coming back `input_required` (which inputs it asked for); an input the client can't answer (no handler registered, or elicitation suppressed for that call); a call stopped for exceeding the 8-round limit | +| `MCP.mrtr` | `2026-07-28` multi-round-trip requests: a call coming back `input_required` (which inputs it asked for); an input the client can't answer (no handler registered, or elicitation suppressed for that call); a call ended because the user cancelled; a server asking again after the user declined, answered with "decline" without prompting; a call stopped for exceeding the 8-round limit | | `MCP.oauth` | the protected-resource metadata fetch; which `.well-known` discovery URL resolved (or that none did); the choice between Dynamic Client Registration, CIMD, and a manual client id — and why CIMD was skipped if it was; the PKCE check; the browser being opened (endpoint, `client_id`, scope); an `iss` mismatch; the code coming back; the token-exchange HTTP status; whether a silent refresh worked; every failure with the server's actual (truncated, redacted) response | | `MCP.stream` | each server-sent-event frame as it's classified — an inbound request (`elicitation/create` and the like), a notification, a result; a server error result; a stream that broke mid-request; a stream that ended without the result it was waiting for | diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index ec045a9..8380466 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -434,7 +434,10 @@ support. Nothing you wrote against the 1.x client changes. `elicitation/create` down an open stream. It returns "input required", and the client asks your handler and re-sends the call with the answer. Your `MCPElicitationHandler` ([§3c](#3c-server-initiated-requests-elicitation-and-the-sampling--roots-seams)) is called - exactly as before, on either revision. Multi-step flows are capped at 8 rounds. + exactly as before, on either revision. Multi-step flows are capped at 8 rounds. Your handler + is asked once per refusal: after a `.cancel` the call ends without re-sending, and after a + `.decline` a server that asks again gets "decline" without your handler being called. Either + way `callTool` returns `.success` with `isError == true` and a sentence the model can read. - **Live updates.** A stateless server that advertises `listChanged` gets one `subscriptions/listen` stream. When its tools, prompts or resources change, the manager re-lists them on its own (enabled flags are kept; a new tool arrives disabled) and From 3485defbfa124778c262f5945fb35fb26e6f6aea Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:03:44 -0700 Subject: [PATCH 11/73] Components (synced): MCPServerPickerView shows server trust and approval (2.0) From locallmlab-sdk: a radio-group auth picker, per-server Trust and "Before a tool runs" approval controls, and notices when none of a server's tools are on or a trusted server's tools changed. Comment citing a private doc scrubbed. Builds against 2.0.0-dev; 39 tests pass. Co-Authored-By: Claude --- .../MCPServerPickerView.swift | 106 +++++++++++++++++- 1 file changed, 104 insertions(+), 2 deletions(-) diff --git a/Components/Sources/LocalLMLabSDKComponents/MCPServerPickerView.swift b/Components/Sources/LocalLMLabSDKComponents/MCPServerPickerView.swift index 684f60a..ebdc9e2 100644 --- a/Components/Sources/LocalLMLabSDKComponents/MCPServerPickerView.swift +++ b/Components/Sources/LocalLMLabSDKComponents/MCPServerPickerView.swift @@ -72,21 +72,35 @@ public struct MCPServerPickerView: View { .frame(width: 140) } + // Radio group, not a segmented control. A segmented `Picker` with 3 long labels + // flips between "hug content" and "fill the frame" distribution across layout passes + // (an empty vs. filled `TextField` reports a different ideal width, and any keystroke + // re-runs layout), so it visibly resized and its label vanished as you typed in the URL + // field. A radio group has no width ambiguity and shows all three options at once. + // Same control as LocalLM Lab's MCP Servers view. Picker("Auth type", selection: $newServerAuthType) { Text("None").tag(MCPAuthType.none) Text("Personal Access Token").tag(MCPAuthType.pat) Text("OAuth (manual client)").tag(MCPAuthType.oauthManual) } - .pickerStyle(.segmented) - .frame(width: 420) + .pickerStyle(.radioGroup) + Text("A server that uses OAuth with dynamic client registration (most hosted servers) needs None: you sign in in your browser when you click Add.") + .font(.system(size: 12)) + .foregroundStyle(.secondary) if newServerAuthType == .pat { SecureField("Personal access token", text: $newServerPATToken) .textFieldStyle(.roundedBorder) + Text("The token's own permissions are what the server gets — this app doesn't narrow them.") + .font(.system(size: 12)) + .foregroundStyle(.secondary) } if newServerAuthType == .oauthManual { TextField("Client ID (from the server's developer console)", text: $newServerManualClientID) .textFieldStyle(.roundedBorder) + Text("For servers with no dynamic client registration (e.g. Slack). Create an app in the server's developer console, opt into PKCE if offered, register \(MCPOAuthFlow.redirectURI) as a redirect URL, then paste the Client ID here — no client secret needed. You'll still sign in in your browser when you click Add.") + .font(.system(size: 12)) + .foregroundStyle(.secondary) } Button(adding ? "Adding…" : "Add") { @@ -136,6 +150,8 @@ public struct MCPServerPickerView: View { .font(.system(size: 12, design: .monospaced)) .foregroundStyle(.secondary) + trustControls(server) + if !server.tools.isEmpty { Text("\(server.tools.filter(\.enabled).count) of \(server.tools.count) tools enabled — ~\(server.estimatedTokens) tokens") .font(.system(size: 12)) @@ -194,6 +210,92 @@ public struct MCPServerPickerView: View { } } + // MARK: trust and approval + + /// The approvals offered here. An approval decided by the app's policy (`by: .policy`) is the + /// app's setting, shown but not offered. + private enum ApprovalChoice: Hashable { + case automatic, allow, askChanges, askDeletions, askEverything, appPolicy + + init(_ approval: ToolApproval?) { + switch approval { + case nil: self = .automatic + case .allow?: self = .allow + case .ask(_, .policy)?: self = .appPolicy + case .ask(.read, .user)?: self = .askEverything + case .ask(.mutate, .user)?: self = .askChanges + case .ask(.destructive, .user)?: self = .askDeletions + case .ask?: self = .askChanges + @unknown default: self = .askChanges + } + } + + var approval: ToolApproval? { + switch self { + case .automatic: nil + case .allow: .allow + case .askChanges: .ask(atOrAbove: .mutate, by: .user) + case .askDeletions: .ask(atOrAbove: .destructive, by: .user) + case .askEverything: .ask(atOrAbove: .read, by: .user) + case .appPolicy: nil + } + } + } + + @ViewBuilder + private func trustControls(_ server: MCPServerState) -> some View { + let id = server.id + HStack(spacing: 16) { + Picker("Trust", selection: Binding( + get: { server.trust }, + set: { manager.core.setTrust($0, server: id) } + )) { + Text("Untrusted").tag(MCPServerTrust.untrusted) + Text("Trusted").tag(MCPServerTrust.trusted) + } + .fixedSize() + Picker("Before a tool runs", selection: Binding( + get: { ApprovalChoice(server.toolApproval) }, + set: { manager.core.setToolApproval($0.approval, server: id) } + )) { + Text(server.trust == .trusted ? "Default: ask before deletions" : "Default: ask before changes") + .tag(ApprovalChoice.automatic) + Text("Don't ask").tag(ApprovalChoice.allow) + Text("Ask before changes").tag(ApprovalChoice.askChanges) + Text("Ask before deletions").tag(ApprovalChoice.askDeletions) + Text("Ask before every call").tag(ApprovalChoice.askEverything) + if ApprovalChoice(server.toolApproval) == .appPolicy { + Text("Decided by the app's policy").tag(ApprovalChoice.appPolicy) + } + } + .fixedSize() + .disabled(ApprovalChoice(server.toolApproval) == .appPolicy) + } + .controlSize(.small) + Text(server.trust == .trusted + ? "Trusted: the server's own labels decide which tools only read and which delete. Trust only operators you vouch for — a server can label anything." + : "Untrusted: the server's labels are ignored, so every tool counts as one that may change data.") + .font(.system(size: 11)) + .foregroundStyle(.secondary) + + if server.toolsChangedSinceTrusted { + HStack(spacing: 8) { + Image(systemName: "exclamationmark.triangle").foregroundStyle(.orange) + Text("This server's tools changed since you trusted it. Review them below.") + Button("Trust Again") { manager.core.setTrust(.trusted, server: id) } + } + .font(.system(size: 12)) + } + if server.connectionStatus == .connected, !server.tools.isEmpty, + !server.tools.contains(where: { $0.enabled && $0.isModelVisible }) { + HStack(spacing: 8) { + Image(systemName: "exclamationmark.circle").foregroundStyle(.orange) + Text("None of this server's tools are on, so a model can't use it yet. Turn on the ones you want below.") + } + .font(.system(size: 12)) + } + } + private func saveSummaryToFile(_ server: MCPServerState) { let panel = NSSavePanel() panel.nameFieldStringValue = "\(server.displayName)-mcp-tools.txt" From 879fdd01127784853877023bf50b6fd6ec49bace Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:03:44 -0700 Subject: [PATCH 12/73] repo-qa-local (synced): run the turn with session.respond, not languageModelSession 2.0: the escape hatch skips the context-overflow retry and the per-turn MCP tool refresh. Same answer text; README diff row updated. Co-Authored-By: Claude --- examples/repo-qa-local/README.md | 2 +- examples/repo-qa-local/Sources/RepoQALocal/main.swift | 6 ++++-- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/examples/repo-qa-local/README.md b/examples/repo-qa-local/README.md index f52aff6..3989d69 100644 --- a/examples/repo-qa-local/README.md +++ b/examples/repo-qa-local/README.md @@ -168,7 +168,7 @@ differences: | `let manager = MCPServerManager()` → `manager.addServer(…)` | `lab.mcp.addServer(…)` — the lab's manager; never a second one | | an `MCPTool(descriptor:manager:)` per tool | `lab.mcp.setToolEnabled(server:tool:enabled: true)` per tool — a new server's tools start **disabled** (security default), so without this the model gets no tools | | `LanguageModelSession(tools: tools) { instructions }` | `lab.makeSession(route: .local, instructions:, includeMCPTools: true)` — builds the `MCPTool`s from `lab.mcp`'s enabled tools | -| `session.respond(to:)` | `session.languageModelSession.respond(to:)` — same FoundationModels session underneath | +| `session.respond(to:)` (a `LanguageModelSession`) | `session.respond(to:)` (a `LocalLMLabSession`) — same FoundationModels session underneath, plus the SDK's context-overflow retry and per-turn MCP tool refresh | That's the point: the model layer is a swap-in, not a rewrite. diff --git a/examples/repo-qa-local/Sources/RepoQALocal/main.swift b/examples/repo-qa-local/Sources/RepoQALocal/main.swift index 2a3e18f..6535be5 100644 --- a/examples/repo-qa-local/Sources/RepoQALocal/main.swift +++ b/examples/repo-qa-local/Sources/RepoQALocal/main.swift @@ -122,9 +122,11 @@ func run() async { let prompt = "Regarding the GitHub repository \"\(repoName)\": \(effectiveQuestion)" note("\nAsking: \(prompt)\n") + // The session's own turn method (repo-qa calls `session.respond(to:)` on its + // LanguageModelSession). Not `session.languageModelSession.respond`: that escape hatch skips + // the context-overflow retry and the per-turn MCP tool refresh. do { - let response = try await session.languageModelSession.respond(to: prompt) - print(response.content) + print(try await session.respond(to: prompt)) } catch { note("Error: \(await GenerationErrorDescription.describe(error))") } From 8d5b7d705ee9f55fed952c11398e2955b850c920 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:03:59 -0700 Subject: [PATCH 13/73] Cascade 2.0.0-dev SDK checksums knownSDKReleases in Components and the 12 other SDK examples (aiql was already done) is now 1.0.0-GA (previous) + 2.0.0-dev; defaultSDKVersion = "2.0.0-dev"; 1.0.0-RC.1 dropped. Each module's checksum is from the v2.0.0-dev release's .sha256 (Core ae20766f, Inference 014bcbf3, Remote 88f8145f, Claude 5986c09b). examples/README.md: this branch builds against 2.0.0-dev, and its knownSDKReleases / .sha256 samples. Built: Components, components-demo, components-updates-demo and every SDK example with swift build (clean .build: an incremental build keeps objects from the 1.0.0-GA interface and fails to link), VistaNova with xcodebuild. Co-Authored-By: Claude --- Components/Package.swift | 10 +++++----- examples/README.md | 16 ++++++++-------- examples/code-buddy/Package.swift | 14 +++++++------- examples/mlx-control-room/Package.swift | 14 +++++++------- examples/model-switch/Package.swift | 10 +++++----- examples/os-matrix/Package.swift | 14 +++++++------- examples/plate-today-tools/Package.swift | 10 +++++----- examples/plate-today/Package.swift | 10 +++++----- examples/repo-qa-local/Package.swift | 14 +++++++------- examples/repo-qa/Package.swift | 10 +++++----- examples/security-demo/Package.swift | 10 +++++----- examples/vistanova/Package.swift | 14 +++++++------- examples/workspace-buddy-local/Package.swift | 14 +++++++------- examples/workspace-buddy/Package.swift | 10 +++++----- 14 files changed, 85 insertions(+), 85 deletions(-) diff --git a/Components/Package.swift b/Components/Package.swift index 7f26faf..bbf0519 100644 --- a/Components/Package.swift +++ b/Components/Package.swift @@ -24,17 +24,17 @@ struct SDKRelease { // or edit `defaultSDKVersion`. For a release not listed, add its entry (URL follows the pattern // below; checksum is the `.sha256` next to the zip on that GitHub release). Keep this in step // with `examples/model-switch/Package.swift`, which consumes Core through this package. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - checksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de" - ), "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), + "2.0.0-dev": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/README.md b/examples/README.md index e85f060..e47ac52 100644 --- a/examples/README.md +++ b/examples/README.md @@ -56,7 +56,7 @@ advanced model-layer features (Private Cloud Compute, open-weight/MLX, Claude, o need it — but it isn't universal: [`os-matrix/`](os-matrix/) is built and tested to run unchanged on **macOS 26** too (see its README). -The examples on this `1.0.0-GA` branch build against SDK **`1.0.0-GA`**; the ones on `main` +The examples on this `2.0.0-dev` branch build against SDK **`2.0.0-dev`** (a pre-release); the ones on `main` build against the latest stable release. No environment variable is needed for either. ### What each example needs @@ -165,13 +165,13 @@ previous one. (`code-buddy/sample-workspace/Package.swift` is not one of these dependency-free fixture for the code-buddy walkthrough, not an SDK consumer.) ```swift -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease(url: "…/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - checksum: "397e7b5f…"), "1.0.0-GA": SDKRelease(url: "…/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2…"), + "2.0.0-dev": SDKRelease(url: "…/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + checksum: "ae20766f…"), ] ``` @@ -182,10 +182,10 @@ every module ships its own checksum (`Core`, `Inference`, `Remote`, and `Claude` distinct `.xcframework.zip.sha256`; don't reuse Core's for another module): ```bash -curl -sL https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip.sha256 -# → 7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e -curl -sL https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKClaude-1.0.0-GA.xcframework.zip.sha256 -# → 2c69c330c8a5e3929b9544fcb865738d0ec39abc7520462cae4f138214e6858f +curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip.sha256 +# → ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9 +curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKClaude-2.0.0-dev.xcframework.zip.sha256 +# → 5986c09b3e7c21b0b5ded462f35ffffd664f54609fda88310b8cfaab111dbf88 ``` Then point `defaultSDKVersion` at it (works everywhere, Xcode included) or pass diff --git a/examples/code-buddy/Package.swift b/examples/code-buddy/Package.swift index 1108bdf..024d117 100644 --- a/examples/code-buddy/Package.swift +++ b/examples/code-buddy/Package.swift @@ -25,21 +25,21 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - coreChecksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKInference-1.0.0-RC.1.xcframework.zip", - inferenceChecksum: "e24cb0581807d37a7b595f0b198b9a9eeecc1c5c36fb59f61429b8a7b42dc169" - ), "1.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", coreChecksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/mlx-control-room/Package.swift b/examples/mlx-control-room/Package.swift index 27141aa..aace797 100644 --- a/examples/mlx-control-room/Package.swift +++ b/examples/mlx-control-room/Package.swift @@ -25,21 +25,21 @@ struct SDKRelease { // its entry (URL + the `.sha256` next to the zip on the GitHub release) or just replace the strings in // place. Needs a release that includes the model pin / update / cleanup APIs (1.0.0-RC.1 as re-published // on 2026-09-18, or later). -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - coreChecksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKInference-1.0.0-RC.1.xcframework.zip", - inferenceChecksum: "e24cb0581807d37a7b595f0b198b9a9eeecc1c5c36fb59f61429b8a7b42dc169" - ), "1.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", coreChecksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/model-switch/Package.swift b/examples/model-switch/Package.swift index 28a1a15..eda39f0 100644 --- a/examples/model-switch/Package.swift +++ b/examples/model-switch/Package.swift @@ -28,17 +28,17 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKRemote-1.0.0-RC.1.xcframework.zip", - remoteChecksum: "3e763c2fb542f7cc8493cf8e4ba2effcc98adae9c7d9991b60e6a92be8ee88c6" - ), "1.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKRemote-1.0.0-GA.xcframework.zip", remoteChecksum: "294a00eb41a22265849de171930c209d48c27c64c157bc565f76a5e0c13bdb88" ), + "2.0.0-dev": SDKRelease( + remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", + remoteChecksum: "88f8145fbaff6ff9d9ec7332abb8837ae4010b2fbbe390527e18ac50fdc77ad3" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/os-matrix/Package.swift b/examples/os-matrix/Package.swift index 00ca69f..3450cc8 100644 --- a/examples/os-matrix/Package.swift +++ b/examples/os-matrix/Package.swift @@ -22,21 +22,21 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - coreChecksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKInference-1.0.0-RC.1.xcframework.zip", - inferenceChecksum: "e24cb0581807d37a7b595f0b198b9a9eeecc1c5c36fb59f61429b8a7b42dc169" - ), "1.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", coreChecksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/plate-today-tools/Package.swift b/examples/plate-today-tools/Package.swift index b91c6cf..ec3b0f1 100644 --- a/examples/plate-today-tools/Package.swift +++ b/examples/plate-today-tools/Package.swift @@ -23,17 +23,17 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - checksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de" - ), "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), + "2.0.0-dev": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/plate-today/Package.swift b/examples/plate-today/Package.swift index 31431a2..7e8751f 100644 --- a/examples/plate-today/Package.swift +++ b/examples/plate-today/Package.swift @@ -20,17 +20,17 @@ struct SDKRelease { // edit `defaultSDKVersion` here. For a release not listed, add its entry — the URL follows the // pattern below and the checksum is the `.sha256` file next to the zip on that GitHub release — // or just replace the two strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - checksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de" - ), "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), + "2.0.0-dev": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index d3ecf11..f098b38 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -26,21 +26,21 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - coreChecksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKInference-1.0.0-RC.1.xcframework.zip", - inferenceChecksum: "e24cb0581807d37a7b595f0b198b9a9eeecc1c5c36fb59f61429b8a7b42dc169" - ), "1.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", coreChecksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/repo-qa/Package.swift b/examples/repo-qa/Package.swift index c4cfade..d540a92 100644 --- a/examples/repo-qa/Package.swift +++ b/examples/repo-qa/Package.swift @@ -23,17 +23,17 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - checksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de" - ), "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), + "2.0.0-dev": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/security-demo/Package.swift b/examples/security-demo/Package.swift index 705f476..75211ef 100644 --- a/examples/security-demo/Package.swift +++ b/examples/security-demo/Package.swift @@ -29,17 +29,17 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKRemote-1.0.0-RC.1.xcframework.zip", - remoteChecksum: "3e763c2fb542f7cc8493cf8e4ba2effcc98adae9c7d9991b60e6a92be8ee88c6" - ), "1.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKRemote-1.0.0-GA.xcframework.zip", remoteChecksum: "294a00eb41a22265849de171930c209d48c27c64c157bc565f76a5e0c13bdb88" ), + "2.0.0-dev": SDKRelease( + remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", + remoteChecksum: "88f8145fbaff6ff9d9ec7332abb8837ae4010b2fbbe390527e18ac50fdc77ad3" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/vistanova/Package.swift b/examples/vistanova/Package.swift index 5b7b78f..6cc3fb5 100644 --- a/examples/vistanova/Package.swift +++ b/examples/vistanova/Package.swift @@ -24,21 +24,21 @@ struct SDKRelease { // strings in place. Needs a release that includes `MLXModelProvider(pinnedRevisions:)` and // `cancelDownload(_:)`: the checksums below are the 1.0.0-RC.1 binaries as re-published on // 2026-09-19. An RC.1 pulled earlier fails the checksum check — re-resolve packages. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - coreChecksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKInference-1.0.0-RC.1.xcframework.zip", - inferenceChecksum: "e24cb0581807d37a7b595f0b198b9a9eeecc1c5c36fb59f61429b8a7b42dc169" - ), "1.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", coreChecksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/workspace-buddy-local/Package.swift b/examples/workspace-buddy-local/Package.swift index 6a2fc63..c45357d 100644 --- a/examples/workspace-buddy-local/Package.swift +++ b/examples/workspace-buddy-local/Package.swift @@ -28,21 +28,21 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - coreChecksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKInference-1.0.0-RC.1.xcframework.zip", - inferenceChecksum: "e24cb0581807d37a7b595f0b198b9a9eeecc1c5c36fb59f61429b8a7b42dc169" - ), "1.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", coreChecksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), ] func failManifest(_ message: String) -> Never { diff --git a/examples/workspace-buddy/Package.swift b/examples/workspace-buddy/Package.swift index 581ddd2..e15f501 100644 --- a/examples/workspace-buddy/Package.swift +++ b/examples/workspace-buddy/Package.swift @@ -21,17 +21,17 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "1.0.0-GA" +let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ - "1.0.0-RC.1": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-RC.1/LocalLMLabSDKCore-1.0.0-RC.1.xcframework.zip", - checksum: "397e7b5f7efd1076293a3d5d06c41d75043bffa23cffdb821d71d21ee41e68de" - ), "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), + "2.0.0-dev": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + ), ] func failManifest(_ message: String) -> Never { From 568f5c4acc85710a2c77d05064d0e95225242c08 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:09:09 -0700 Subject: [PATCH 14/73] MCPAppsHost (2.0): host-side MCP Apps support, shipped as source From locallmlab-sdk: `ui://` widget metadata and resource loading, MCPAppsBridge (the widget<->host JSON-RPC state machine, with policy and audit sink), the sandboxed MCPAppViewController / MCPAppView, MCPAppsSessionBackend (a widget's calls through the LocalLMLabSession), and the widget lifecycle (MCPAppWidgetCache, MCPAppViewPool, MCPAppRecreation). Tests included. Package.swift rewritten for the public tree: Core comes from ../Components (which declares the binary target and re-vends it), so no checksum of its own. Comments citing private design docs, phases and prototype IDs scrubbed. Builds against 2.0.0-dev; 85 tests pass (same as the SDK repo). Co-Authored-By: Claude --- MCPAppsHost/Package.swift | 38 ++ .../MCPAppMetadata.swift | 160 ++++++ .../MCPAppRecreation.swift | 168 ++++++ .../MCPAppViewController.swift | 328 +++++++++++ .../MCPAppViewPool.swift | 98 ++++ .../MCPAppWidgetCache.swift | 115 ++++ .../MCPAppsBridge.swift | 512 ++++++++++++++++++ .../MCPAppsBridgePolicy.swift | 108 ++++ .../MCPAppsPages.swift | 125 +++++ .../MCPAppsSandbox.swift | 172 ++++++ .../MCPAppsSessionBackend.swift | 81 +++ .../MCPToolInputForm.swift | 171 ++++++ .../MCPAppLifecycleTests.swift | 235 ++++++++ .../MCPAppMetadataTests.swift | 109 ++++ .../MCPAppViewControllerTests.swift | 225 ++++++++ .../MCPAppsBridgeTests.swift | 438 +++++++++++++++ .../MCPAppsSandboxTests.swift | 156 ++++++ .../MCPAppsSessionBackendTests.swift | 45 ++ .../MCPToolInputFormTests.swift | 123 +++++ 19 files changed, 3407 insertions(+) create mode 100644 MCPAppsHost/Package.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppMetadata.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppRecreation.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewController.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewPool.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppWidgetCache.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridge.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridgePolicy.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsPages.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSandbox.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSessionBackend.swift create mode 100644 MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPToolInputForm.swift create mode 100644 MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppLifecycleTests.swift create mode 100644 MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppMetadataTests.swift create mode 100644 MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppViewControllerTests.swift create mode 100644 MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsBridgeTests.swift create mode 100644 MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSandboxTests.swift create mode 100644 MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSessionBackendTests.swift create mode 100644 MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPToolInputFormTests.swift diff --git a/MCPAppsHost/Package.swift b/MCPAppsHost/Package.swift new file mode 100644 index 0000000..9758d48 --- /dev/null +++ b/MCPAppsHost/Package.swift @@ -0,0 +1,38 @@ +// swift-tools-version: 6.0 +import PackageDescription + +// LocalLMLabSDKMCPAppsHost — host-side support for MCP Apps (SEP-1865, spec 2026-01-26): the +// `ui://` widgets an MCP server ships and a host renders in a sandboxed view. Open source +// (Apache 2.0), shipped as SOURCE like Components, and built on LocalLMLabSDKCore's PUBLIC API +// only. +// +// Core comes from Components, which declares the LocalLMLabSDKCore binary target and re-vends it +// (two packages declaring that binary target is a hard SwiftPM error). So this package has no +// SDK version or checksum of its own: it builds against whatever Components' `defaultSDKVersion` +// / LOCALLM_SDK_VERSION selects. Requires SDK 2.0.0-dev or later. +// +// What is here: metadata parsing (`_meta.ui`) and the `ui://` resource loader (mime check, size, +// SHA-256); `MCPAppsBridge`, the JSON-RPC state machine between a widget and the host, with an +// explicit policy and an audit sink; `MCPAppViewController` / `MCPAppView`, the sandboxed +// WKWebView; `MCPAppsSessionBackend`, which routes a widget's calls through a LocalLMLabSession; +// and the widget lifecycle — `MCPAppWidgetCache`, `MCPAppViewPool`, `MCPAppRecreation`. +let package = Package( + name: "LocalLMLabSDKMCPAppsHost", + platforms: [.macOS("26.0")], + products: [ + .library(name: "LocalLMLabSDKMCPAppsHost", targets: ["LocalLMLabSDKMCPAppsHost"]) + ], + dependencies: [ + .package(path: "../Components") + ], + targets: [ + .target( + name: "LocalLMLabSDKMCPAppsHost", + dependencies: [.product(name: "LocalLMLabSDKCore", package: "Components")] + ), + .testTarget( + name: "LocalLMLabSDKMCPAppsHostTests", + dependencies: ["LocalLMLabSDKMCPAppsHost"] + ) + ] +) diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppMetadata.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppMetadata.swift new file mode 100644 index 0000000..32f0757 --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppMetadata.swift @@ -0,0 +1,160 @@ +import CryptoKit +import Foundation +import LocalLMLabSDKCore + +// `_meta.ui` parsing for MCP Apps (SEP-1865). Everything here reads the verbatim `_meta` +// `MCPValue` Core surfaces on tool descriptors and resource contents; nothing is trusted — a +// malformed or hostile `_meta` degrades to "no UI" / "no permissions", never to a wider grant. + +// Tool-side MCP Apps parsing (`MCPToolDescriptor.app`, `.appVisibility`, `.isModelVisible`, +// `MCPAppVisibility`, `MCPAppLink`) lives in LocalLMLabSDKCore, so every host applies the same +// visibility rules. This file keeps the resource side: what a fetched `ui://` widget declares. + +/// The CSP origins a `ui://` resource asks for (`_meta.ui.csp`). +public struct MCPAppCSP: Sendable, Equatable { + public var connectDomains: [String] = [] + public var resourceDomains: [String] = [] + public var frameDomains: [String] = [] + public var baseUriDomains: [String] = [] + + public init(connectDomains: [String] = [], resourceDomains: [String] = [], frameDomains: [String] = [], baseUriDomains: [String] = []) { + self.connectDomains = connectDomains + self.resourceDomains = resourceDomains + self.frameDomains = frameDomains + self.baseUriDomains = baseUriDomains + } + + /// True when the resource asks for no origins at all (Todoist's widget declares exactly this). + public var isEmpty: Bool { + connectDomains.isEmpty && resourceDomains.isEmpty && frameDomains.isEmpty && baseUriDomains.isEmpty + } +} + +/// Device permissions a `ui://` resource asks for (`_meta.ui.permissions`). +public struct MCPAppPermissions: Sendable, Equatable { + public var camera = false + public var microphone = false + public var geolocation = false + public var clipboardWrite = false + + public init(camera: Bool = false, microphone: Bool = false, geolocation: Bool = false, clipboardWrite: Bool = false) { + self.camera = camera + self.microphone = microphone + self.geolocation = geolocation + self.clipboardWrite = clipboardWrite + } + + public var isEmpty: Bool { !(camera || microphone || geolocation || clipboardWrite) } +} + +/// What a `ui://` resource declares about itself in `_meta.ui`. These are a server's *requests*; +/// the host decides what to grant (intersect with policy, never widen). +public struct MCPAppResourceInfo: Sendable, Equatable { + public var csp: MCPAppCSP + public var permissions: MCPAppPermissions + public var domain: String? + public var prefersBorder: Bool? + + public init(csp: MCPAppCSP = MCPAppCSP(), permissions: MCPAppPermissions = MCPAppPermissions(), domain: String? = nil, prefersBorder: Bool? = nil) { + self.csp = csp + self.permissions = permissions + self.domain = domain + self.prefersBorder = prefersBorder + } + + /// Parses a resource's `_meta`. `nil` or malformed input yields the empty (most restrictive) + /// declaration. + public init(meta: MCPValue?) { + self.init() + guard case .object(let top)? = meta, case .object(let ui)? = top["ui"] else { return } + func strings(_ value: MCPValue?) -> [String] { + guard case .array(let items)? = value else { return [] } + return items.compactMap { if case .string(let s) = $0 { return s } else { return nil } } + } + if case .object(let csp)? = ui["csp"] { + self.csp = MCPAppCSP( + connectDomains: strings(csp["connectDomains"]), + resourceDomains: strings(csp["resourceDomains"]), + frameDomains: strings(csp["frameDomains"]), + baseUriDomains: strings(csp["baseUriDomains"])) + } + if case .object(let perms)? = ui["permissions"] { + func has(_ key: String) -> Bool { perms[key] != nil && perms[key] != .bool(false) && perms[key] != .null } + permissions = MCPAppPermissions( + camera: has("camera"), microphone: has("microphone"), + geolocation: has("geolocation"), clipboardWrite: has("clipboardWrite")) + } + if case .string(let d)? = ui["domain"] { domain = d } + if case .bool(let b)? = ui["prefersBorder"] { prefersBorder = b } + } +} + +public enum MCPAppResourceError: Error, Equatable, CustomStringConvertible { + case notAUIResource(String) + case wrongMimeType(String?) + case noBody + case invalidBase64 + + public var description: String { + switch self { + case .notAUIResource(let uri): return "\(uri) is not a ui:// resource" + case .wrongMimeType(let m): return "expected text/html;profile=mcp-app, got \(m ?? "no mimeType")" + case .noBody: return "resource has neither text nor blob" + case .invalidBase64: return "resource blob is not valid base64" + } + } +} + +/// A fetched `ui://` widget: its HTML, what it declared, and the SHA-256 of the exact bytes — the +/// value an IT allowlist pins. +public struct MCPAppResource: Sendable, Equatable { + public static let mimeType = "text/html;profile=mcp-app" + + public let uri: String + public let html: String + public let info: MCPAppResourceInfo + /// Lowercase hex SHA-256 of the UTF-8 HTML. + public let sha256: String + + /// Validates and wraps a `resources/read` result. `listedMeta` is the `_meta` from + /// `resources/list`; the content entry's own `_meta` wins when both exist (spec: the read + /// result is authoritative). + public init(content: MCPResourceContent, requestedURI: String, listedMeta: MCPValue? = nil) throws { + guard requestedURI.hasPrefix("ui://") else { throw MCPAppResourceError.notAUIResource(requestedURI) } + let normalizedMime = content.mimeType?.lowercased().filter { !$0.isWhitespace } + guard normalizedMime == Self.mimeType else { throw MCPAppResourceError.wrongMimeType(content.mimeType) } + let html: String + if let text = content.text { + html = text + } else if let blob = content.blob { + guard let data = Data(base64Encoded: blob), let decoded = String(data: data, encoding: .utf8) else { + throw MCPAppResourceError.invalidBase64 + } + html = decoded + } else { + throw MCPAppResourceError.noBody + } + self.uri = requestedURI // never the server-echoed URI + self.html = html + self.info = MCPAppResourceInfo(meta: content.meta ?? listedMeta) + self.sha256 = SHA256.hash(data: Data(html.utf8)).map { String(format: "%02x", $0) }.joined() + } +} + +extension MCPClientHandlers { + /// The client capability key under which a host declares MCP Apps support at `initialize`. + public static let mcpAppsExtensionKey = "io.modelcontextprotocol/ui" + + /// These handlers, declaring MCP Apps support (`text/html;profile=mcp-app` widgets). Servers + /// that gate their widgets on it (`_meta.ui` on tools) only offer them to a client that says so. + /// + /// ```swift + /// let lab = LocalLMLab(configuration: .init( + /// providers: [...], mcp: MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps()))) + /// ``` + public func advertisingMCPApps() -> MCPClientHandlers { + var handlers = self + handlers.extensions[Self.mcpAppsExtensionKey] = .object(["mimeTypes": .array([.string(MCPAppResource.mimeType)])]) + return handlers + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppRecreation.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppRecreation.swift new file mode 100644 index 0000000..3fca79c --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppRecreation.swift @@ -0,0 +1,168 @@ +import Foundation +import FoundationModels +import LocalLMLabSDKCore + +// Re-creating a widget from a stored tool call: +// show the cached version at once, then check the server in the background and decide what to do +// if its widget changed in between. The decision is a pure function, so it is tested without +// WebKit or a server. + +/// What a host should show for a re-created widget. +public enum MCPAppRecreationDecision: Sendable, Equatable { + /// Show `MCPAppRecreation.Revalidation.resource` with the stored result. `viewOnly`: the server + /// could not be reached, so the widget's tool calls would fail — present it as view-only. + case render(viewOnly: Bool) + /// The tool's output changed shape since the call: the stored result may not fit the current + /// widget. Show the snapshot (or a placeholder) with a refresh action that re-runs the call. + /// `mayRefreshAutomatically`: the tool is annotated read-only, so re-running it is safe without + /// asking; otherwise ask first. + case needsRefresh(mayRefreshAutomatically: Bool) + /// A deployment's hash pin does not approve the widget version that would be shown. + case notApproved + /// Nothing cached and the server could not be reached. + case unavailable +} + +public enum MCPAppRecreation { + /// The result of checking a stored call against its server. + public struct Revalidation: Sendable { + public var decision: MCPAppRecreationDecision + /// The widget version to show for `.render`; `nil` otherwise. + public var resource: MCPAppResource? + /// True when the server serves a different widget version than the call was rendered + /// with. + public var widgetChanged: Bool + } + + /// Decides what to show. Rules: + /// - server reachable, same version → the current widget; + /// - server reachable, newer version, tool output schema unchanged → the current widget (it + /// carries the vendor's fixes) with the stored result; + /// - server reachable, newer (or unknown) version, output schema changed → refresh; + /// - server unreachable → the cached version, view-only (or `.unavailable` if none); + /// - a pin (`approvedSHA256`) that does not include the version to show → `.notApproved`. + public static func decide(cachedSHA256: String?, currentSHA256: String?, outputSchemaUnchanged: Bool, + toolIsReadOnly: Bool, approvedSHA256: Set? = nil) -> MCPAppRecreationDecision { + func approved(_ sha: String) -> Bool { approvedSHA256.map { $0.contains(sha) } ?? true } + guard let current = currentSHA256 else { + guard let cached = cachedSHA256 else { return .unavailable } + return approved(cached) ? .render(viewOnly: true) : .notApproved + } + // A version we cannot vouch for (changed, or unknown because it was never cached or was + // evicted) only gets the stored result if the result's shape still fits. + if current != cachedSHA256, !outputSchemaUnchanged { + return .needsRefresh(mayRefreshAutomatically: toolIsReadOnly) + } + return approved(current) ? .render(viewOnly: false) : .notApproved + } + + /// The cached widget a stored call was rendered with, for showing at once (before + /// `revalidate`). `nil` if it was never cached or has been evicted. + @MainActor + public static func cachedResource(recordID: String, cache: MCPAppWidgetCache) -> MCPAppResource? { + cache.sha256(forRecord: recordID).flatMap { cache.resource(sha256: $0) } + } + + /// Checks the server's current widget and tool against a stored call and decides what to + /// show. Fetches and caches the current widget when the server is reachable. + /// + /// - Parameters: + /// - callDescriptor: the tool's descriptor when the call was made + /// (`ToolCallRecord.toolDescriptor`), for the output-schema comparison. + @MainActor + public static func revalidate(recordID: String, resourceURI: String, callDescriptor: MCPToolDescriptor?, + source: any MCPAppWidgetSource, cache: MCPAppWidgetCache, + approvedSHA256: Set? = nil) async -> Revalidation { + let cachedSHA = cache.sha256(forRecord: recordID) + let current: MCPAppResource? + if let content = try? await source.currentResource(uri: resourceURI).get() { + current = try? cache.store(content, requestedURI: resourceURI) + } else { + current = nil + } + let currentDescriptor = callDescriptor.flatMap { source.currentDescriptor(tool: $0.name) } + let schemaUnchanged = sameJSON(callDescriptor?.outputSchema, currentDescriptor?.outputSchema) + let readOnly = (currentDescriptor ?? callDescriptor)?.annotations?.isReadOnly ?? false + let decision = decide(cachedSHA256: cachedSHA, currentSHA256: current?.sha256, + outputSchemaUnchanged: schemaUnchanged, toolIsReadOnly: readOnly, + approvedSHA256: approvedSHA256) + var resource: MCPAppResource? + if case .render(let viewOnly) = decision { + resource = viewOnly ? cachedSHA.flatMap { cache.resource(sha256: $0) } : current + if let current, !viewOnly { cache.bind(recordID: recordID, sha256: current.sha256) } + } + return Revalidation(decision: decision, resource: resource, + widgetChanged: current != nil && cachedSHA != nil && current?.sha256 != cachedSHA) + } + + /// `ToolCallRecord` conveniences: a record has a widget to re-create when it has an MCP App + /// link and a stored result. + @MainActor + public static func revalidate(_ record: ToolCallRecord, source: any MCPAppWidgetSource, cache: MCPAppWidgetCache, + approvedSHA256: Set? = nil) async -> Revalidation? { + guard let uri = record.app?.resourceURI, record.mcpResult != nil else { return nil } + return await revalidate(recordID: record.id, resourceURI: uri, callDescriptor: record.toolDescriptor, + source: source, cache: cache, approvedSHA256: approvedSHA256) + } + + /// Hands a stored call to a (re-created) widget: its arguments as `tool-input` and its result as + /// `tool-result` — what the widget got the first time, with no call to the server. The bridge + /// holds them until the widget is ready. + @MainActor + public static func deliver(arguments: GeneratedContent?, result: MCPToolResult, to controller: MCPAppViewController) { + controller.bridge.deliverToolInput(arguments: toolInput(from: arguments)) + controller.bridge.deliverToolResult(result) + } + + /// `deliver(arguments:result:to:)` for a `ToolCallRecord`. Returns `false` (and delivers + /// nothing) when the record has no stored result. + @MainActor + @discardableResult + public static func deliver(_ record: ToolCallRecord, to controller: MCPAppViewController) -> Bool { + guard let result = record.mcpResult else { return false } + deliver(arguments: record.arguments, result: result, to: controller) + return true + } + + /// A tool call's arguments in the form a widget receives them (`ui/notifications/tool-input`). + public static func toolInput(from arguments: GeneratedContent?) -> [String: MCPValue] { + guard let json = arguments?.jsonString, let data = json.data(using: .utf8), + case .object(let object)? = try? JSONDecoder().decode(MCPValue.self, from: data) else { return [:] } + return object + } + + static func sameJSON(_ a: Data?, _ b: Data?) -> Bool { + func parse(_ d: Data?) -> NSObject? { d.flatMap { try? JSONSerialization.jsonObject(with: $0) as? NSObject } } + let x = parse(a), y = parse(b) + if x == nil, y == nil { return a == b } + return x?.isEqual(y) ?? false + } +} + +/// Where re-creation reads the server's current widget and tool. +@MainActor +public protocol MCPAppWidgetSource { + func currentResource(uri: String) async -> Result + func currentDescriptor(tool: String) -> MCPToolDescriptor? +} + +/// The standard source: one server of an `MCPServerManager`. +@MainActor +public struct MCPServerManagerWidgetSource: MCPAppWidgetSource { + let manager: MCPServerManager + let server: MCPServerID + + public init(manager: MCPServerManager, server: MCPServerID) { + self.manager = manager + self.server = server + } + + public func currentResource(uri: String) async -> Result { + guard manager.servers[server]?.connectionStatus == .connected else { return .failure(.notConnected) } + return await manager.readResource(server: server, uri: uri) + } + + public func currentDescriptor(tool: String) -> MCPToolDescriptor? { + manager.servers[server]?.tools.first { $0.name == tool } + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewController.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewController.swift new file mode 100644 index 0000000..a559d19 --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewController.swift @@ -0,0 +1,328 @@ +import AppKit +import Foundation +import LocalLMLabSDKCore +import SwiftUI +import WebKit + +/// Owns one widget's `WKWebView` and the `MCPAppsBridge` behind it. +/// +/// Layers of containment, outermost first (each holds even if the one above it failed): +/// 1. the effective CSP (declared ∩ host policy), served as a header on the widget document; +/// 2. a WKContentRuleList blocking every http(s)/ws(s)/ftp load outside the effective origins; +/// 3. the widget runs in a sandboxed iframe (opaque origin) inside a relay page; +/// 4. navigation is pinned to this view's own `mcpapp:///` documents; new windows, file +/// pickers and media/geolocation capture are refused; +/// 5. the native message handler accepts only main-frame (relay) messages, and every message +/// goes through `MCPAppsBridge`'s policy. +/// The web view uses a non-persistent data store: no cookies, cache or storage survive it. +@MainActor +public final class MCPAppViewController: NSObject, ObservableObject { + public enum LoadState: Equatable { + case idle, loading, loaded + case failed(String) + } + + public let webView: WKWebView + /// The bridge to deliver `tool-input` / `tool-result` through. Available immediately; the + /// bridge queues notifications until the widget reports it is initialized. + public private(set) var bridge: MCPAppsBridge! + /// The CSP actually enforced for this view. + public let effectiveCSP: MCPAppCSP + public let resource: MCPAppResource + @Published public private(set) var loadState: LoadState = .idle + /// Height the widget last asked for (`ui/notifications/size-changed`), for SwiftUI layout. + @Published public private(set) var contentHeight: CGFloat? + /// The display mode in effect: `"inline"` (fills the space the host gives the view) or + /// `"fullscreen"` (the host UI should expand the view over its chrome). Changes when the widget + /// asks for a mode listed in `supportedDisplayModes`, or when `exitFullscreen()` is called. + @Published public private(set) var displayMode: String = "inline" + /// The modes this controller offers the widget; a widget shows a fullscreen button only if + /// `"fullscreen"` is here. List it only if the host UI reacts to `displayMode`. + public let supportedDisplayModes: [String] + + private let pages: MCPAppsPages + private let sandboxPolicy: MCPAppsSandboxPolicy + private let ruleListIdentifier: String + private var ruleList: WKContentRuleList? + private var closed = false + /// The widget iframe's frame, captured when its document is navigated to (testing hook only). + private var widgetFrame: WKFrameInfo? + private var appearanceObservation: NSKeyValueObservation? + private var frameObserver: NSObjectProtocol? + private var resizeTask: Task? + private var lastReportedSize: CGSize = .zero + + /// - Parameters: + /// - resource: the fetched, validated widget. (Pin/verify `resource.sha256` BEFORE creating + /// the controller; the controller renders whatever it is given.) + /// - tools: the bound server's tools, for the bridge's tool-call checks. + public init( + resource: MCPAppResource, + tools: [MCPToolDescriptor], + backend: any MCPAppsBackend, + configuration: MCPAppsBridgeConfiguration, + bridgePolicy: any MCPAppsBridgePolicy = MCPAppsDefaultPolicy(), + sandboxPolicy: MCPAppsSandboxPolicy = .closed, + actions: MCPAppsHostActions = MCPAppsHostActions(), + supportedDisplayModes: [String] = ["inline"], + audit: @escaping @MainActor (MCPAppsAuditEvent) -> Void = { _ in } + ) { + let effective = MCPAppsSandbox.effectiveCSP(declared: resource.info.csp, policy: sandboxPolicy) + self.effectiveCSP = effective + self.resource = resource + self.sandboxPolicy = sandboxPolicy + self.pages = MCPAppsPages(widgetHTML: resource.html, csp: effective, allowsSameOrigin: sandboxPolicy.allowsSameOrigin) + self.ruleListIdentifier = "mcp-app-" + pages.token + + let config = WKWebViewConfiguration() + config.websiteDataStore = .nonPersistent() + config.defaultWebpagePreferences.allowsContentJavaScript = true + config.preferences.javaScriptCanOpenWindowsAutomatically = false + config.mediaTypesRequiringUserActionForPlayback = .all + let handler = MCPAppsSchemeHandler(pages: pages) + config.setURLSchemeHandler(handler, forURLScheme: MCPAppsPages.scheme) + let webView = WKWebView(frame: .zero, configuration: config) + webView.isInspectable = sandboxPolicy.isInspectable + webView.allowsMagnification = false + webView.allowsBackForwardNavigationGestures = false + webView.setValue(false, forKey: "drawsBackground") + self.webView = webView + self.supportedDisplayModes = supportedDisplayModes.contains("inline") ? supportedDisplayModes : ["inline"] + supportedDisplayModes + + super.init() + + // Report exactly what this controller enforces, not what the server asked for. + var configuration = configuration + configuration.effectiveCSP = effective + configuration.effectivePermissions = MCPAppPermissions() // no device permissions are granted + configuration.availableDisplayModes = self.supportedDisplayModes + if configuration.hostContext["theme"] == nil { configuration.hostContext["theme"] = .string(Self.theme(of: webView)) } + var actions = actions + let userRequestDisplayMode = actions.requestDisplayMode + actions.requestDisplayMode = { [weak self] mode in + guard let self, self.supportedDisplayModes.contains(mode) else { return false } + _ = userRequestDisplayMode(mode) + self.displayMode = mode + return true + } + self.bridge = MCPAppsBridge( + configuration: configuration, tools: tools, backend: backend, policy: bridgePolicy, + send: { [weak webView] data in + guard let webView, let json = String(data: data, encoding: .utf8) else { return } + webView.evaluateJavaScript("window.__mcpDeliver(\(Self.jsStringLiteral(json)))", completionHandler: nil) + }, + audit: audit, + onSizeChanged: { [weak self] _, height in + guard let height, height.isFinite, height > 0 else { return } + self?.contentHeight = min(CGFloat(height), 4000) + }, + actions: actions) + + // The handler holds the controller weakly: WKUserContentController retains handlers. + let relay = WeakScriptHandler(owner: self) + config.userContentController.add(relay, name: "mcpBridge") + webView.navigationDelegate = self + webView.uiDelegate = self + observeHostContext() + } + + /// Leaves fullscreen (e.g. the host's Esc / close button) and tells the widget. + public func exitFullscreen() { + guard displayMode != "inline" else { return } + displayMode = "inline" + bridge.setDisplayMode("inline") + } + + // MARK: host context the widget adapts to + + private static func theme(of view: NSView) -> String { + view.effectiveAppearance.bestMatch(from: [.aqua, .darkAqua]) == .darkAqua ? "dark" : "light" + } + + /// Keeps the widget's `hostContext` current: light/dark follows the system, and the container + /// size follows the web view (debounced, and only when it actually changed). + private func observeHostContext() { + appearanceObservation = webView.observe(\.effectiveAppearance, options: [.new]) { [weak self] view, _ in + MainActor.assumeIsolated { + guard let self else { return } + self.bridge.deliverHostContextChange(["theme": .string(Self.theme(of: view))]) + } + } + webView.postsFrameChangedNotifications = true + frameObserver = NotificationCenter.default.addObserver(forName: NSView.frameDidChangeNotification, object: webView, queue: .main) { [weak self] _ in + MainActor.assumeIsolated { self?.scheduleSizeReport() } + } + } + + private func scheduleSizeReport() { + resizeTask?.cancel() + resizeTask = Task { [weak self] in + try? await Task.sleep(for: .milliseconds(120)) + guard !Task.isCancelled, let self, !self.closed else { return } + let size = self.webView.bounds.size + guard size.width > 0, size.height > 0, size != self.lastReportedSize else { return } + self.lastReportedSize = size + self.bridge.deliverHostContextChange([ + "containerDimensions": .object(["width": .number(Double(size.width)), "height": .number(Double(size.height))]) + ]) + } + } + + /// Compiles the network block list, then loads the relay page. Resolves when the load has + /// been started; `loadState` reports completion. + public func load() async { + guard !closed, loadState == .idle else { return } + loadState = .loading + do { + let json = MCPAppsSandbox.contentRuleListJSON(for: effectiveCSP) + guard let list = try await WKContentRuleListStore.default().compileContentRuleList( + forIdentifier: ruleListIdentifier, encodedContentRuleList: json) else { + loadState = .failed("Could not compile the network block list") + return // fail closed: never load a widget without the network block in place + } + ruleList = list + webView.configuration.userContentController.add(list) + } catch { + loadState = .failed("Could not compile the network block list: \(error.localizedDescription)") + return + } + webView.load(URLRequest(url: pages.hostURL)) + } + + /// Tears the view down: asks the widget to clean up, closes the bridge, removes the message + /// handler and the block list. Call when the view goes away. + /// + /// Closing a widget means `close()` **and releasing** this controller and its `webView`: the + /// widget's WebContent process (~100 MB) ends only when the web view is deallocated, not when + /// `close()` returns (measured). `MCPAppViewPool` does both. + public func close() async { + guard !closed else { return } + closed = true + resizeTask?.cancel() + appearanceObservation?.invalidate() + appearanceObservation = nil + if let frameObserver { NotificationCenter.default.removeObserver(frameObserver) } + frameObserver = nil + await bridge.teardown() + webView.stopLoading() + webView.configuration.userContentController.removeScriptMessageHandler(forName: "mcpBridge") + webView.configuration.userContentController.removeAllContentRuleLists() + try? await WKContentRuleListStore.default().removeContentRuleList(forIdentifier: ruleListIdentifier) + ruleList = nil + } + + /// Runs `script` inside the WIDGET's frame with native privilege (it does not need, and is not + /// subject to, the frame's opaque origin). For automated tests and diagnostics only — e.g. + /// clicking a control in a widget to prove a click reaches the server through the bridge. Not + /// public API: import with `@_spi(Testing)`. + @_spi(Testing) + public func evaluateInWidget(_ script: String) async throws -> Any? { + guard let widgetFrame else { throw URLError(.badURL) } + return try await webView.evaluateJavaScript(script, in: widgetFrame, contentWorld: .page) + } + + /// A JS string literal for arbitrary text (JSON string encoding is valid JS; U+2028/2029 are + /// legal in string literals since ES2019). + static func jsStringLiteral(_ text: String) -> String { + guard let data = try? JSONSerialization.data(withJSONObject: [text], options: []), + let array = String(data: data, encoding: .utf8) else { return "\"\"" } + return String(array.dropFirst().dropLast()) + } +} + +// MARK: - message handler + +private final class WeakScriptHandler: NSObject, WKScriptMessageHandler { + weak var owner: MCPAppViewController? + + init(owner: MCPAppViewController) { + self.owner = owner + } + + func userContentController(_ userContentController: WKUserContentController, didReceive message: WKScriptMessage) { + MainActor.assumeIsolated { + owner?.handle(message) + } + } +} + +extension MCPAppViewController { + fileprivate func handle(_ message: WKScriptMessage) { + // Only the relay page (main frame) may talk to native code. The widget lives in a + // subframe; if it reaches `webkit.messageHandlers` directly its message is dropped here. + guard message.frameInfo.isMainFrame, message.frameInfo.request.url?.host == pages.token else { return } + guard let body = message.body as? String else { return } + let data = Data(body.utf8) + // One Task per message: MainActor tasks start in FIFO order, and the handshake handlers + // don't suspend before replying, so ordering of initialize -> initialized is preserved. + Task { await bridge.receive(data) } + } +} + +// MARK: - navigation + UI containment + +extension MCPAppViewController: WKNavigationDelegate { + public func webView(_ webView: WKWebView, decidePolicyFor navigationAction: WKNavigationAction) async -> WKNavigationActionPolicy { + // Pin every navigation (main frame, iframe, link click, redirect) to this view's own + // documents. A link the widget wants opened goes through `ui/open-link` and the policy. + guard let url = navigationAction.request.url, url.scheme == MCPAppsPages.scheme, url.host == pages.token, + url.path == MCPAppsPages.hostPath || url.path == MCPAppsPages.widgetPath else { + return .cancel + } + if url.path == MCPAppsPages.widgetPath { widgetFrame = navigationAction.targetFrame } + return .allow + } + + public func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) { + if loadState == .loading { loadState = .loaded } + } + + public func webView(_ webView: WKWebView, didFail navigation: WKNavigation!, withError error: any Error) { + loadState = .failed(error.localizedDescription) + } + + public func webView(_ webView: WKWebView, didFailProvisionalNavigation navigation: WKNavigation!, withError error: any Error) { + loadState = .failed(error.localizedDescription) + } +} + +extension MCPAppViewController: WKUIDelegate { + public func webView(_ webView: WKWebView, createWebViewWith configuration: WKWebViewConfiguration, for navigationAction: WKNavigationAction, windowFeatures: WKWindowFeatures) -> WKWebView? { + nil // no popups, no target=_blank + } + + public func webView(_ webView: WKWebView, runJavaScriptAlertPanelWithMessage message: String, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable () -> Void) { + completionHandler() // widgets don't get to put up native dialogs + } + + public func webView(_ webView: WKWebView, runJavaScriptConfirmPanelWithMessage message: String, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable (Bool) -> Void) { + completionHandler(false) + } + + public func webView(_ webView: WKWebView, runJavaScriptTextInputPanelWithPrompt prompt: String, defaultText: String?, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable (String?) -> Void) { + completionHandler(nil) + } + + public func webView(_ webView: WKWebView, runOpenPanelWith parameters: WKOpenPanelParameters, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable ([URL]?) -> Void) { + completionHandler(nil) // no file access + } + + public func webView(_ webView: WKWebView, requestMediaCapturePermissionFor origin: WKSecurityOrigin, initiatedByFrame frame: WKFrameInfo, type: WKMediaCaptureType, decisionHandler: @escaping @MainActor @Sendable (WKPermissionDecision) -> Void) { + decisionHandler(.deny) // camera / microphone: never granted (see `effectivePermissions`) + } +} + +// MARK: - SwiftUI + +/// Hosts a controller's web view in SwiftUI. The owner keeps the controller alive and calls +/// `load()` once and `close()` when done. +public struct MCPAppView: NSViewRepresentable { + private let controller: MCPAppViewController + + public init(controller: MCPAppViewController) { + self.controller = controller + } + + public func makeNSView(context: Context) -> WKWebView { controller.webView } + public func updateNSView(_ nsView: WKWebView, context: Context) {} +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewPool.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewPool.swift new file mode 100644 index 0000000..8755ade --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppViewPool.swift @@ -0,0 +1,98 @@ +import AppKit +import Foundation +import WebKit + +/// Keeps at most `limit` widgets live. +/// +/// Each live widget is a `WKWebView` with its own WebContent process — about 105 MB each, as +/// measured in a chat host. Past the limit the least recently used widget is snapshotted, closed and +/// **released** (`close()` alone does not end its process; dropping the view does), and the host +/// shows `snapshot(for:)` in its place until the user scrolls back or taps it, when the host asks +/// for the view again and it is re-created. +@MainActor +public final class MCPAppViewPool: ObservableObject { + /// How many widgets stay live. Default 3. Lowering it releases widgets right away. + public var limit: Int { + didSet { trim() } + } + + @Published public private(set) var liveKeys: [String] = [] + @Published private var snapshots: [String: NSImage] = [:] + private var controllers: [String: MCPAppViewController] = [:] + + public init(limit: Int = 3) { + self.limit = max(1, limit) + } + + /// The live controller for `key` (e.g. the tool-call record id), creating it with `make` if it + /// is not live, and marking it most recently used. `make` is not called for a live key. + public func controller(for key: String, make: () throws -> MCPAppViewController) rethrows -> MCPAppViewController { + if let live = controllers[key] { + touch(key) + return live + } + let controller = try make() + controllers[key] = controller + touch(key) + trim() + return controller + } + + /// Marks `key` as most recently used (e.g. it scrolled into view). + public func touch(_ key: String) { + guard controllers[key] != nil else { return } + liveKeys.removeAll { $0 == key } + liveKeys.append(key) + } + + public func isLive(_ key: String) -> Bool { controllers[key] != nil } + + /// The picture of a released widget, shown in its place. `nil` while it is live or if it was + /// released before it could be captured. + public func snapshot(for key: String) -> NSImage? { snapshots[key] } + + /// Closes and releases `key`'s widget, keeping a snapshot. + public func release(_ key: String) async { + guard let controller = detach(key) else { return } + await finish(key, controller) + } + + /// Stops treating `key` as live, at once. + private func detach(_ key: String) -> MCPAppViewController? { + guard let controller = controllers.removeValue(forKey: key) else { return nil } + liveKeys.removeAll { $0 == key } + return controller + } + + private func finish(_ key: String, _ controller: MCPAppViewController) async { + if let image = await Self.snapshot(controller.webView) { snapshots[key] = image } + await controller.close() + controller.webView.removeFromSuperview() + } + + /// Releases every widget (e.g. the conversation closed). + public func releaseAll() async { + for key in liveKeys { await release(key) } + } + + /// Forgets a snapshot (e.g. its entry was deleted). + public func discardSnapshot(for key: String) { snapshots.removeValue(forKey: key) } + + private func trim() { + let excess = liveKeys.count - limit + guard excess > 0 else { return } + // Detach now so the pool never reports an evicted widget as live; snapshot and close + // finish asynchronously. + for key in Array(liveKeys.prefix(excess)) { + guard let controller = detach(key) else { continue } + Task { await finish(key, controller) } + } + } + + private static func snapshot(_ webView: WKWebView) async -> NSImage? { + guard webView.bounds.width > 0, webView.bounds.height > 0 else { return nil } + return await withCheckedContinuation { continuation in + webView.takeSnapshot(with: nil) { image, _ in continuation.resume(returning: image) } + } + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppWidgetCache.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppWidgetCache.swift new file mode 100644 index 0000000..856eb48 --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppWidgetCache.swift @@ -0,0 +1,115 @@ +import Foundation +import LocalLMLabSDKCore + +/// Widget HTML kept on disk by content hash, so a widget can be shown again — after scrolling back +/// past the live-widget limit, after a relaunch, or offline — without fetching it, and so a host can +/// tell when a server's widget changed. +/// +/// One file per widget **version** (SHA-256 of its HTML), not per call: a widget is 1.6–3.2 MB +/// (measured), and many calls render the same version. The cache also remembers which version each tool +/// call was rendered with (`bind(recordID:sha256:)`), because a record alone does not say. Least +/// recently used versions are evicted past `maxBytes`. +@MainActor +public final class MCPAppWidgetCache { + public let directory: URL + /// Upper bound on the HTML kept on disk. Default 64 MB (about 20–40 widget versions). + public var maxBytes: Int { + didSet { evict() } + } + + private struct Stored: Codable { + var requestedURI: String + var content: MCPResourceContent + } + + private struct Index: Codable { + var recordVersion: [String: String] = [:] + var lastUsed: [String: Date] = [:] + var size: [String: Int] = [:] + } + + private var index: Index + + /// - Parameter directory: where to keep the cache. Default: `Caches/LocalLMLab/MCPAppWidgets` + /// in the app's container. + public init(directory: URL? = nil, maxBytes: Int = 64 * 1024 * 1024) { + let base = directory ?? FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0] + .appendingPathComponent("LocalLMLab/MCPAppWidgets", isDirectory: true) + self.directory = base + self.maxBytes = maxBytes + try? FileManager.default.createDirectory(at: base, withIntermediateDirectories: true) + index = (try? Data(contentsOf: base.appendingPathComponent("index.json"))) + .flatMap { try? JSONDecoder().decode(Index.self, from: $0) } ?? Index() + } + + /// Validates `content` as an MCP App (`MCPAppResource`), keeps it, and returns it. + @discardableResult + public func store(_ content: MCPResourceContent, requestedURI: String) throws -> MCPAppResource { + let resource = try MCPAppResource(content: content, requestedURI: requestedURI) + let data = try JSONEncoder().encode(Stored(requestedURI: requestedURI, content: content)) + try data.write(to: file(resource.sha256), options: .atomic) + index.size[resource.sha256] = data.count + index.lastUsed[resource.sha256] = Date() + evict() + save() + return resource + } + + /// The cached widget version with this hash, or `nil`. + public func resource(sha256: String) -> MCPAppResource? { + guard let data = try? Data(contentsOf: file(sha256)), + let stored = try? JSONDecoder().decode(Stored.self, from: data), + let resource = try? MCPAppResource(content: stored.content, requestedURI: stored.requestedURI), + resource.sha256 == sha256 else { return nil } // a damaged or swapped file is not served + index.lastUsed[sha256] = Date() + save() + return resource + } + + /// Remembers that tool call `recordID` was rendered with widget version `sha256`. + public func bind(recordID: String, sha256: String) { + index.recordVersion[recordID] = sha256 + save() + } + + /// The widget version tool call `recordID` was rendered with, if known. + public func sha256(forRecord recordID: String) -> String? { index.recordVersion[recordID] } + + public var storedBytes: Int { index.size.values.reduce(0, +) } + public var storedVersions: Set { Set(index.size.keys) } + + /// Fetches the widget a tool links to from its server, keeps it, and binds it to the call. + public func fetch(resourceURI: String, server: MCPServerID, manager: MCPServerManager, + recordID: String? = nil) async throws -> MCPAppResource { + let content = try await manager.readResource(server: server, uri: resourceURI).get() + let listed = manager.servers[server]?.resources.first { $0.uri == resourceURI }?.meta + let merged = content.meta == nil && listed != nil + ? MCPResourceContent(uri: content.uri, mimeType: content.mimeType, text: content.text, blob: content.blob, meta: listed) + : content + let resource = try store(merged, requestedURI: resourceURI) + if let recordID { bind(recordID: recordID, sha256: resource.sha256) } + return resource + } + + // MARK: private + + private func file(_ sha256: String) -> URL { + let safe = sha256.filter { $0.isHexDigit } + return directory.appendingPathComponent("\(safe).json") + } + + private func evict() { + while storedBytes > maxBytes, index.size.count > 1, + let oldest = index.size.keys.min(by: { (index.lastUsed[$0] ?? .distantPast) < (index.lastUsed[$1] ?? .distantPast) }) { + try? FileManager.default.removeItem(at: file(oldest)) + index.size.removeValue(forKey: oldest) + index.lastUsed.removeValue(forKey: oldest) + } + } + + private func save() { + if let data = try? JSONEncoder().encode(index) { + try? data.write(to: directory.appendingPathComponent("index.json"), options: .atomic) + } + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridge.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridge.swift new file mode 100644 index 0000000..0bb4e59 --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridge.swift @@ -0,0 +1,512 @@ +import Foundation +import LocalLMLabSDKCore + +/// Host settings a bridge reports to its view at `ui/initialize`. +public struct MCPAppsBridgeConfiguration: Sendable { + public var hostName: String + public var hostVersion: String + /// Extra `hostContext` fields (theme, styles, locale, timeZone, containerDimensions, + /// toolInfo, ...). `displayMode`, `availableDisplayModes`, and `platform` get defaults if absent. + public var hostContext: [String: MCPValue] + /// What the host will actually enforce for this view, advertised under + /// `hostCapabilities.sandbox.csp`. Should be the effective (intersected) CSP, not the + /// server's request. + public var effectiveCSP: MCPAppCSP + public var effectivePermissions: MCPAppPermissions + /// Display modes the host can actually provide, advertised in `hostContext`. A widget only + /// offers a fullscreen button if `"fullscreen"` is listed here, so list only what the host UI + /// implements. `"inline"` is always present. + public var availableDisplayModes: [String] + /// Incoming messages larger than this are dropped (and audited), unparsed. + public var maxMessageBytes: Int + /// How long `teardown` waits for the view to answer `ui/resource-teardown`. + public var teardownTimeout: Duration + + public init( + hostName: String, hostVersion: String, + hostContext: [String: MCPValue] = [:], + effectiveCSP: MCPAppCSP = MCPAppCSP(), + effectivePermissions: MCPAppPermissions = MCPAppPermissions(), + availableDisplayModes: [String] = ["inline"], + maxMessageBytes: Int = 4 * 1024 * 1024, + teardownTimeout: Duration = .seconds(2) + ) { + self.hostName = hostName + self.hostVersion = hostVersion + self.hostContext = hostContext + self.effectiveCSP = effectiveCSP + self.effectivePermissions = effectivePermissions + self.availableDisplayModes = availableDisplayModes.contains("inline") ? availableDisplayModes : ["inline"] + availableDisplayModes + self.maxMessageBytes = maxMessageBytes + self.teardownTimeout = teardownTimeout + } +} + +/// What the host DOES once the policy has allowed a view's request. The bridge decides whether a +/// request may proceed; these closures carry it out. All default to no-ops, so a host that +/// keeps the closed default policy never needs to set them. +public struct MCPAppsHostActions { + /// Open an http(s) URL (already scheme-checked by the bridge). + public var openLink: @MainActor (URL) -> Void + /// A view's `ui/message`: put this user-role text into the conversation. + public var sendMessage: @MainActor (String) -> Void + /// A view's `ui/update-model-context`: replace what the model knows about the view's state. + public var updateModelContext: @MainActor (_ content: [MCPValue], _ structuredContent: MCPValue?) -> Void + /// A view's `ui/request-display-mode` for a mode the host advertised. Return `true` if the + /// host UI switched to it (the bridge then tells the view via `host-context-changed`); `false` + /// leaves the mode unchanged. + public var requestDisplayMode: @MainActor (_ mode: String) -> Bool + + public init( + openLink: @escaping @MainActor (URL) -> Void = { _ in }, + sendMessage: @escaping @MainActor (String) -> Void = { _ in }, + updateModelContext: @escaping @MainActor (_ content: [MCPValue], _ structuredContent: MCPValue?) -> Void = { _, _ in }, + requestDisplayMode: @escaping @MainActor (_ mode: String) -> Bool = { _ in false } + ) { + self.openLink = openLink + self.sendMessage = sendMessage + self.updateModelContext = updateModelContext + self.requestDisplayMode = requestDisplayMode + } +} + +/// The JSON-RPC state machine between ONE view (a `ui://` widget) and the host — pure Swift, no +/// WebKit. A web-view layer feeds it raw messages via `receive(_:)` and ships whatever it emits +/// through `send`. A host with no model uses it as-is; a host with a model wires the opt-in +/// policy methods. +/// +/// Guarantees (each covered by tests): +/// - nothing is sent to the view before it sends `ui/notifications/initialized` (spec); host +/// notifications requested earlier are queued and flushed in order; +/// - a view request before initialization, a second `ui/initialize`, or any request after +/// teardown is refused; +/// - `tools/call` reaches only tools the server declared, whose visibility includes `app`, that +/// the policy approves, on the ONE server the backend is bound to; +/// - `ui/message`, `ui/update-model-context`, `ui/open-link` and non-`ui://` reads go through the +/// policy, which denies them by default; +/// - malformed, oversized, or unknown input is dropped or answered with a JSON-RPC error, never +/// crashes and never widens access; +/// - every message in either direction leaves an audit event without argument values or output. +@MainActor +public final class MCPAppsBridge { + public enum State: Sendable, Equatable { + case awaitingInitialize + case awaitingInitialized + case ready + case tornDown + } + + /// The MCP Apps spec revision this bridge speaks. + public static let protocolVersion = "2026-01-26" + + public private(set) var state: State = .awaitingInitialize + /// The display mode currently in effect (`"inline"` or `"fullscreen"`). + public private(set) var displayMode: String + + private let configuration: MCPAppsBridgeConfiguration + private let tools: [String: MCPToolDescriptor] + private let backend: any MCPAppsBackend + private let policy: any MCPAppsBridgePolicy + private let send: @MainActor (Data) -> Void + private let audit: @MainActor (MCPAppsAuditEvent) -> Void + private let onSizeChanged: @MainActor (_ width: Double?, _ height: Double?) -> Void + private let actions: MCPAppsHostActions + + private var queued: [[String: Any]] = [] + /// `host-context-changed` fields requested before the view is ready, merged into ONE + /// notification (later values win) so a burst of resizes doesn't become a burst of messages. + private var pendingContext: [String: MCPValue] = [:] + private var teardownContinuation: CheckedContinuation? + private static let teardownRequestID = "host-teardown-1" + + /// - Parameters: + /// - tools: the bound server's tool list (`MCPServerState.tools`); the bridge only ever + /// lets the view call a tool that appears here. + /// - send: delivers one serialized JSON-RPC message to the view. + public init( + configuration: MCPAppsBridgeConfiguration, + tools: [MCPToolDescriptor], + backend: any MCPAppsBackend, + policy: any MCPAppsBridgePolicy = MCPAppsDefaultPolicy(), + send: @escaping @MainActor (Data) -> Void, + audit: @escaping @MainActor (MCPAppsAuditEvent) -> Void = { _ in }, + onSizeChanged: @escaping @MainActor (_ width: Double?, _ height: Double?) -> Void = { _, _ in }, + actions: MCPAppsHostActions = MCPAppsHostActions() + ) { + self.configuration = configuration + self.tools = Dictionary(tools.map { ($0.name, $0) }, uniquingKeysWith: { first, _ in first }) + self.backend = backend + self.policy = policy + self.send = send + self.audit = audit + self.onSizeChanged = onSizeChanged + self.actions = actions + if case .string(let mode)? = configuration.hostContext["displayMode"], configuration.availableDisplayModes.contains(mode) { + self.displayMode = mode + } else { + self.displayMode = "inline" + } + } + + // MARK: - Host -> view + + /// `ui/notifications/tool-input`: the arguments of the tool call the view is rendering. + public func deliverToolInput(arguments: [String: MCPValue]) { + emit(notification: "ui/notifications/tool-input", params: ["arguments": Self.jsonObject(.object(arguments))]) + } + + /// `ui/notifications/tool-result`: the tool's result — `structuredContent` and `_meta` are + /// what a widget renders from. + public func deliverToolResult(_ result: MCPToolResult) { + emit(notification: "ui/notifications/tool-result", params: Self.resultObject(result)) + } + + /// `ui/notifications/tool-cancelled`. + public func deliverToolCancelled(reason: String? = nil) { + emit(notification: "ui/notifications/tool-cancelled", params: reason.map { ["reason": $0] } ?? [:]) + } + + /// `ui/notifications/host-context-changed`: only the fields that changed (theme, size, ...). + public func deliverHostContextChange(_ partial: [String: MCPValue]) { + guard state != .tornDown, !partial.isEmpty else { return } + if state != .ready { + pendingContext.merge(partial) { _, new in new } + return + } + emit(notification: "ui/notifications/host-context-changed", params: Self.jsonObject(.object(partial)) as? [String: Any] ?? [:]) + } + + /// The HOST changed the display mode itself (e.g. the user left fullscreen). Updates the mode + /// and tells the view. Ignored for a mode the host didn't advertise. + public func setDisplayMode(_ mode: String) { + guard configuration.availableDisplayModes.contains(mode), mode != displayMode else { return } + displayMode = mode + deliverHostContextChange(["displayMode": .string(mode)]) + } + + /// Asks the view to clean up (`ui/resource-teardown`), waits up to `teardownTimeout` for its + /// answer, then closes the bridge. Always closes, answered or not. + public func teardown(reason: String = "host closing") async { + guard state != .tornDown else { return } + if state == .ready || state == .awaitingInitialized { + let message: [String: Any] = [ + "jsonrpc": "2.0", "id": Self.teardownRequestID, + "method": "ui/resource-teardown", "params": ["reason": reason], + ] + transmit(message, method: "ui/resource-teardown") + await withCheckedContinuation { (continuation: CheckedContinuation) in + teardownContinuation = continuation + let timeout = configuration.teardownTimeout + Task { [weak self] in + try? await Task.sleep(for: timeout) + self?.finishTeardownWait() + } + } + } + state = .tornDown + queued.removeAll() + } + + private func finishTeardownWait() { + guard let continuation = teardownContinuation else { return } + teardownContinuation = nil + continuation.resume() + } + + // MARK: - View -> host + + /// Feed one raw message from the view (a JSON-RPC 2.0 object). + public func receive(_ data: Data) async { + guard state != .tornDown else { + audit(.init(direction: .fromView, method: "?", outcome: .dropped, detail: "after teardown")) + return + } + guard data.count <= configuration.maxMessageBytes else { + audit(.init(direction: .fromView, method: "?", outcome: .dropped, detail: "oversized message")) + return + } + guard let object = (try? JSONSerialization.jsonObject(with: data)) as? [String: Any], + (object["jsonrpc"] as? String) == "2.0" else { + audit(.init(direction: .fromView, method: "?", outcome: .dropped, detail: "not JSON-RPC 2.0")) + return + } + if let method = object["method"] as? String { + let params = object["params"] as? [String: Any] ?? [:] + if let id = object["id"], Self.isValidID(id) { + await handleRequest(id: id, method: method, params: params) + } else if object["id"] == nil { + handleNotification(method: method, params: params) + } else { + audit(.init(direction: .fromView, method: method, outcome: .dropped, detail: "invalid id")) + } + } else if let id = object["id"] as? String, id == Self.teardownRequestID, + object["result"] != nil || object["error"] != nil { + audit(.init(direction: .fromView, method: "ui/resource-teardown", outcome: .ok)) + finishTeardownWait() + } else { + audit(.init(direction: .fromView, method: "?", outcome: .dropped, detail: "unrecognized message")) + } + } + + /// JSON-RPC ids are strings or numbers. `id is Bool` is NOT a usable check on a + /// JSONSerialization result — `NSNumber(1)` bridges to `Bool` — so test the CF type instead. + private static func isValidID(_ id: Any) -> Bool { + if id is String { return true } + guard let number = id as? NSNumber else { return false } + return CFGetTypeID(number) != CFBooleanGetTypeID() + } + + private func handleNotification(method: String, params: [String: Any]) { + switch method { + case "ui/notifications/initialized": + guard state == .awaitingInitialized else { + audit(.init(direction: .fromView, method: method, outcome: .dropped, detail: "unexpected in state \(state)")) + return + } + state = .ready + audit(.init(direction: .fromView, method: method, outcome: .ok)) + let pending = queued + queued.removeAll() + for message in pending { transmit(message, method: (message["method"] as? String) ?? "?") } + if !pendingContext.isEmpty { + let merged = pendingContext + pendingContext.removeAll() + deliverHostContextChange(merged) + } + case "ui/notifications/size-changed": + onSizeChanged((params["width"] as? NSNumber)?.doubleValue, (params["height"] as? NSNumber)?.doubleValue) + audit(.init(direction: .fromView, method: method, outcome: .ok)) + case "notifications/message": + // Logging from the view: recorded as an event only, never its text. + audit(.init(direction: .fromView, method: method, outcome: .ok)) + default: + audit(.init(direction: .fromView, method: method, outcome: .dropped, detail: "unknown notification")) + } + } + + private func handleRequest(id: Any, method: String, params: [String: Any]) async { + switch method { + case "ui/initialize": + guard state == .awaitingInitialize else { + return fail(id, method, .invalidRequest("Already initialized")) + } + guard params["protocolVersion"] is String else { + return fail(id, method, .invalidParams("protocolVersion is required")) + } + state = .awaitingInitialized + reply(id, method, result: initializeResult()) + case "ping": + reply(id, method, result: [:]) + case "tools/call": + await handleToolCall(id: id, params: params) + case "resources/read": + await handleResourceRead(id: id, params: params) + case "ui/open-link": + guard state == .ready else { return fail(id, method, .notReady) } + guard let raw = params["url"] as? String, let url = URL(string: raw), + let scheme = url.scheme?.lowercased(), scheme == "https" || scheme == "http" else { + return fail(id, method, .server("Invalid URL")) + } + switch await policy.authorizeOpenLink(url) { + case .allow: + actions.openLink(url) + reply(id, method, result: [:]) + case .deny(let reason): fail(id, method, .server(reason), outcome: .denied) + } + case "ui/message": + guard state == .ready else { return fail(id, method, .notReady) } + guard let content = params["content"] as? [String: Any], let text = content["text"] as? String else { + return fail(id, method, .server("Invalid message format")) + } + switch await policy.authorizeMessage(text: text) { + case .allow: + actions.sendMessage(text) + reply(id, method, result: [:]) + case .deny(let reason): fail(id, method, .server(reason), outcome: .denied) + } + case "ui/update-model-context": + guard state == .ready else { return fail(id, method, .notReady) } + switch await policy.authorizeModelContextUpdate() { + case .allow: + let content: [MCPValue] = (params["content"] as? [Any])?.compactMap(Self.mcpValue) ?? [] + actions.updateModelContext(content, (params["structuredContent"]).flatMap(Self.mcpValue)) + reply(id, method, result: [:]) + case .deny(let reason): fail(id, method, .server(reason), outcome: .denied) + } + case "ui/request-display-mode": + guard state == .ready else { return fail(id, method, .notReady) } + // The host owns layout. A request for an unadvertised mode, or one the host UI + // declines, is answered with the mode actually in effect (not an error). + if let requested = params["mode"] as? String, requested != displayMode, + configuration.availableDisplayModes.contains(requested), actions.requestDisplayMode(requested) { + displayMode = requested + deliverHostContextChange(["displayMode": .string(requested)]) + } + reply(id, method, result: ["mode": displayMode]) + default: + fail(id, method, .methodNotFound) + } + } + + private func handleToolCall(id: Any, params: [String: Any]) async { + let method = "tools/call" + guard state == .ready else { return fail(id, method, .notReady) } + guard let name = params["name"] as? String else { return fail(id, method, .invalidParams("name is required")) } + var arguments: [String: MCPValue] = [:] + if let raw = params["arguments"] { + guard let object = raw as? [String: Any], let decoded = Self.mcpValue(object), case .object(let a) = decoded else { + return fail(id, method, .invalidParams("arguments must be an object")) + } + arguments = a + } + guard let descriptor = tools[name] else { + return fail(id, method, .server("Unknown tool"), detail: name, outcome: .denied) + } + guard descriptor.appVisibility.contains(.app) else { + return fail(id, method, .server("Tool is not callable by apps"), detail: name, outcome: .denied) + } + if case .deny(let reason) = await policy.authorizeToolCall(name: name, arguments: arguments) { + return fail(id, method, .server(reason), detail: name, outcome: .denied) + } + switch await backend.callTool(name: name, arguments: arguments) { + case .success(let result): + reply(id, method, result: Self.resultObject(result), detail: name) + case .failure(let error): + fail(id, method, .server("Tool call failed (\(Self.errorClass(error)))"), detail: name) + } + } + + private func handleResourceRead(id: Any, params: [String: Any]) async { + let method = "resources/read" + guard state == .ready else { return fail(id, method, .notReady) } + guard let uri = params["uri"] as? String else { return fail(id, method, .invalidParams("uri is required")) } + if case .deny(let reason) = await policy.authorizeResourceRead(uri: uri) { + return fail(id, method, .server(reason), outcome: .denied) + } + switch await backend.readResource(uri: uri) { + case .success(let content): + var entry: [String: Any] = ["uri": content.uri] + if let mime = content.mimeType { entry["mimeType"] = mime } + if let text = content.text { entry["text"] = text } + if let blob = content.blob { entry["blob"] = blob } + if let meta = content.meta { entry["_meta"] = Self.jsonObject(meta) } + reply(id, method, result: ["contents": [entry]]) + case .failure(let error): + fail(id, method, .server("Resource read failed (\(Self.errorClass(error)))")) + } + } + + // MARK: - Building messages + + private func initializeResult() -> [String: Any] { + var caps: [String: Any] = ["serverTools": [String: Any](), "serverResources": [String: Any]()] + var sandbox: [String: Any] = [ + "csp": [ + "connectDomains": configuration.effectiveCSP.connectDomains, + "resourceDomains": configuration.effectiveCSP.resourceDomains, + "frameDomains": configuration.effectiveCSP.frameDomains, + "baseUriDomains": configuration.effectiveCSP.baseUriDomains, + ] + ] + var perms: [String: Any] = [:] + let p = configuration.effectivePermissions + if p.camera { perms["camera"] = [String: Any]() } + if p.microphone { perms["microphone"] = [String: Any]() } + if p.geolocation { perms["geolocation"] = [String: Any]() } + if p.clipboardWrite { perms["clipboardWrite"] = [String: Any]() } + sandbox["permissions"] = perms + caps["sandbox"] = sandbox + + var context = (Self.jsonObject(.object(configuration.hostContext)) as? [String: Any]) ?? [:] + context["displayMode"] = displayMode + context["availableDisplayModes"] = configuration.availableDisplayModes + if context["platform"] == nil { context["platform"] = "desktop" } + + return [ + "protocolVersion": Self.protocolVersion, + "hostInfo": ["name": configuration.hostName, "version": configuration.hostVersion], + "hostCapabilities": caps, + "hostContext": context, + ] + } + + private static func resultObject(_ result: MCPToolResult) -> [String: Any] { + var object: [String: Any] = [:] + object["content"] = result.text.isEmpty ? [[String: Any]]() : [["type": "text", "text": result.text]] + if let structured = result.structuredContent { object["structuredContent"] = jsonObject(structured) } + if result.isError { object["isError"] = true } + if let meta = result.meta { object["_meta"] = jsonObject(meta) } + return object + } + + private enum RPCError { + case invalidRequest(String), invalidParams(String), methodNotFound, notReady, server(String) + + var code: Int { + switch self { + case .invalidRequest: return -32600 + case .methodNotFound: return -32601 + case .invalidParams: return -32602 + case .notReady: return -32600 + case .server: return -32000 + } + } + var message: String { + switch self { + case .invalidRequest(let m), .invalidParams(let m), .server(let m): return m + case .methodNotFound: return "Method not found" + case .notReady: return "Not initialized" + } + } + } + + private func reply(_ id: Any, _ method: String, result: [String: Any], detail: String? = nil) { + transmit(["jsonrpc": "2.0", "id": id, "result": result], method: method, isReplyTo: method, outcome: .ok, detail: detail) + } + + private func fail(_ id: Any, _ method: String, _ error: RPCError, detail: String? = nil, outcome: MCPAppsAuditEvent.Outcome = .error) { + transmit( + ["jsonrpc": "2.0", "id": id, "error": ["code": error.code, "message": error.message]], + method: method, isReplyTo: method, outcome: outcome, detail: detail ?? error.message) + } + + private func emit(notification method: String, params: [String: Any]) { + guard state != .tornDown else { return } + let message: [String: Any] = ["jsonrpc": "2.0", "method": method, "params": params] + if state == .ready { transmit(message, method: method) } else { queued.append(message) } + } + + private func transmit(_ message: [String: Any], method: String, isReplyTo: String? = nil, outcome: MCPAppsAuditEvent.Outcome = .ok, detail: String? = nil) { + guard JSONSerialization.isValidJSONObject(message), let data = try? JSONSerialization.data(withJSONObject: message) else { + audit(.init(direction: .toView, method: method, outcome: .error, detail: "unserializable")) + return + } + send(data) + audit(.init(direction: isReplyTo == nil ? .toView : .fromView, method: method, outcome: outcome, detail: detail)) + } + + // MARK: - JSON <-> MCPValue (public Codable only; Core's Any bridges are internal) + + private static func jsonObject(_ value: MCPValue) -> Any { + guard let data = try? JSONEncoder().encode(value), + let object = try? JSONSerialization.jsonObject(with: data, options: [.fragmentsAllowed]) else { return NSNull() } + return object + } + + private static func mcpValue(_ object: Any) -> MCPValue? { + guard JSONSerialization.isValidJSONObject(object) || object is String || object is NSNumber, + let data = try? JSONSerialization.data(withJSONObject: object, options: [.fragmentsAllowed]) else { return nil } + return try? JSONDecoder().decode(MCPValue.self, from: data) + } + + private static func errorClass(_ error: MCPServerError) -> String { + switch error { + case .unreachable: return "unreachable" + case .notConnected: return "not connected" + case .authorizationRequired, .credentialRejected: return "not authorized" + case .toolNotFound: return "unknown tool" + case .responseTooLarge: return "response too large" + default: return "server error" + } + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridgePolicy.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridgePolicy.swift new file mode 100644 index 0000000..bac9f2a --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsBridgePolicy.swift @@ -0,0 +1,108 @@ +import Foundation +import LocalLMLabSDKCore + +/// The server-side of a bridge: what a view's `tools/call` and `resources/read` are proxied to. +/// A backend is bound to ONE MCP server, so a view can never reach another server's tools — the +/// spec's "same server only" rule holds by construction, not by a check. +@MainActor +public protocol MCPAppsBackend { + func callTool(name: String, arguments: [String: MCPValue]) async -> Result + func readResource(uri: String) async -> Result +} + +/// The standard backend: one server of an `MCPServerManager`. +@MainActor +public struct MCPServerManagerBackend: MCPAppsBackend { + private let manager: MCPServerManager + private let server: MCPServerID + + public init(manager: MCPServerManager, server: MCPServerID) { + self.manager = manager + self.server = server + } + + public func callTool(name: String, arguments: [String: MCPValue]) async -> Result { + // Elicitation stays available: a view-initiated call is still the user's session. + await manager.callTool(server: server, tool: name, arguments: arguments) + } + + public func readResource(uri: String) async -> Result { + await manager.readResource(server: server, uri: uri) + } +} + +public enum MCPAppsDecision: Sendable, Equatable { + case allow + case deny(String) +} + +/// The host's say over what a view may do, on top of what the protocol itself enforces (tool +/// visibility, initialization order). A view is untrusted third-party JavaScript: every effect it +/// requests passes through here. +/// +/// Defaults are closed except for tool calls the spec already scopes to app-visible tools of the +/// bound server: `ui/message`, `ui/update-model-context`, `ui/open-link` and non-`ui://` resource +/// reads are DENIED unless the host opts in. A host with no model in the loop simply keeps the +/// defaults. +@MainActor +public protocol MCPAppsBridgePolicy { + /// Called after the protocol checks (known tool, `app` in its visibility). This is where an + /// enterprise host confirms writes, applies a per-tool allowlist, or rate-limits. + func authorizeToolCall(name: String, arguments: [String: MCPValue]) async -> MCPAppsDecision + func authorizeResourceRead(uri: String) async -> MCPAppsDecision + func authorizeOpenLink(_ url: URL) async -> MCPAppsDecision + /// `ui/message`: the view asks to put a user message into the conversation. Only meaningful + /// in a hosted-model shape (B/C). + func authorizeMessage(text: String) async -> MCPAppsDecision + /// `ui/update-model-context`: the view asks to change what the model knows. + func authorizeModelContextUpdate() async -> MCPAppsDecision +} + +extension MCPAppsBridgePolicy { + public func authorizeToolCall(name: String, arguments: [String: MCPValue]) async -> MCPAppsDecision { .allow } + public func authorizeResourceRead(uri: String) async -> MCPAppsDecision { + uri.hasPrefix("ui://") ? .allow : .deny("Only ui:// resources may be read by a view") + } + public func authorizeOpenLink(_ url: URL) async -> MCPAppsDecision { .deny("Link opening denied by policy") } + public func authorizeMessage(text: String) async -> MCPAppsDecision { .deny("Message sending denied by policy") } + public func authorizeModelContextUpdate() async -> MCPAppsDecision { .deny("Context update denied by policy") } +} + +/// The closed-by-default policy: exactly the protocol defaults, with an optional tool allowlist. +@MainActor +public struct MCPAppsDefaultPolicy: MCPAppsBridgePolicy { + /// When non-nil, only these tools may be called by a view (an IT-approved list). `nil` means + /// every app-visible tool of the bound server. + public var allowedTools: Set? + + public init(allowedTools: Set? = nil) { + self.allowedTools = allowedTools + } + + public func authorizeToolCall(name: String, arguments: [String: MCPValue]) async -> MCPAppsDecision { + if let allowedTools, !allowedTools.contains(name) { return .deny("Tool \(name) is not approved for apps") } + return .allow + } +} + +/// One line of the bridge's audit trail. Argument values and tool output are never recorded — +/// only method names, tool names and outcomes — so the log is safe to persist. +public struct MCPAppsAuditEvent: Sendable, Equatable { + /// `fromView`: a view request/notification and how the host answered it. `toView`: a + /// host-initiated notification or request. + public enum Direction: String, Sendable { case fromView, toView } + public enum Outcome: String, Sendable { case ok, denied, error, dropped } + + public let direction: Direction + public let method: String + public let outcome: Outcome + /// A short, non-sensitive note (tool name, denial reason, error class). + public let detail: String? + + public init(direction: Direction, method: String, outcome: Outcome, detail: String? = nil) { + self.direction = direction + self.method = method + self.outcome = outcome + self.detail = detail + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsPages.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsPages.swift new file mode 100644 index 0000000..6ea75e3 --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsPages.swift @@ -0,0 +1,125 @@ +import Foundation +import WebKit + +/// The two documents a widget view serves from the private `mcpapp:///` scheme: +/// +/// - `host.html`: a tiny relay page (main frame) that embeds the widget in a sandboxed iframe and +/// shuttles JSON-RPC messages between that iframe and native code. This is the spec's native +/// arrangement: the widget sees a normal `window.parent`, but it runs in an opaque origin and +/// cannot reach the native message handler (which the controller accepts from the main frame +/// only). +/// - `widget.html`: the server's HTML, served with the effective CSP as a response header. +/// +/// Anything else 404s. The token is unguessable per view, so another view (or page) cannot +/// address this one's documents. +struct MCPAppsPages { + static let scheme = "mcpapp" + static let hostPath = "/host.html" + static let widgetPath = "/widget.html" + + let token: String + let nonce: String + let widgetHTML: String + let csp: MCPAppCSP + let allowsSameOrigin: Bool + + init(widgetHTML: String, csp: MCPAppCSP, allowsSameOrigin: Bool) { + // 128 bits of randomness each, hex: safe as a hostname label and as a CSP nonce. + func random() -> String { (0..<16).map { _ in String(format: "%02x", UInt8.random(in: .min ... .max)) }.joined() } + self.token = random() + self.nonce = random() + self.widgetHTML = widgetHTML + self.csp = csp + self.allowsSameOrigin = allowsSameOrigin + } + + var hostURL: URL { URL(string: "\(Self.scheme)://\(token)\(Self.hostPath)")! } + var widgetURL: URL { URL(string: "\(Self.scheme)://\(token)\(Self.widgetPath)")! } + + struct Page { + var body: Data + var headers: [String: String] + } + + /// The document for `url`, or `nil` (404) for a wrong scheme, host token, or path. + func page(for url: URL) -> Page? { + guard url.scheme == Self.scheme, url.host == token else { return nil } + let common = [ + "Content-Type": "text/html; charset=utf-8", + "X-Content-Type-Options": "nosniff", + "Referrer-Policy": "no-referrer", + "Cache-Control": "no-store", + ] + switch url.path { + case Self.hostPath: + var headers = common + headers["Content-Security-Policy"] = hostCSP + return Page(body: Data(hostHTML.utf8), headers: headers) + case Self.widgetPath: + var headers = common + headers["Content-Security-Policy"] = MCPAppsSandbox.headerValue(for: csp) + return Page(body: Data(widgetHTML.utf8), headers: headers) + default: + return nil + } + } + + /// The relay page can only run its own nonce'd script and frame the widget document. + var hostCSP: String { + "default-src 'none'; script-src 'nonce-\(nonce)'; style-src 'nonce-\(nonce)'; " + + "frame-src \(Self.scheme)://\(token); base-uri 'none'; form-action 'none'; object-src 'none'" + } + + var hostHTML: String { + let sandbox = allowsSameOrigin ? "allow-scripts allow-forms allow-same-origin" : "allow-scripts allow-forms" + return """ + + + + + + + + """ + } +} + +/// Serves `MCPAppsPages` to WebKit. Requests for anything but this view's two documents fail. +@MainActor +final class MCPAppsSchemeHandler: NSObject, WKURLSchemeHandler { + private let pages: MCPAppsPages + + init(pages: MCPAppsPages) { + self.pages = pages + } + + func webView(_ webView: WKWebView, start urlSchemeTask: any WKURLSchemeTask) { + let url = urlSchemeTask.request.url + guard let url, urlSchemeTask.request.httpMethod == "GET" || urlSchemeTask.request.httpMethod == nil, + let page = pages.page(for: url), + let response = HTTPURLResponse(url: url, statusCode: 200, httpVersion: "HTTP/1.1", headerFields: page.headers) else { + urlSchemeTask.didFailWithError(URLError(.fileDoesNotExist)) + return + } + urlSchemeTask.didReceive(response) + urlSchemeTask.didReceive(page.body) + urlSchemeTask.didFinish() + } + + func webView(_ webView: WKWebView, stop urlSchemeTask: any WKURLSchemeTask) {} +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSandbox.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSandbox.swift new file mode 100644 index 0000000..1e2e7cb --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSandbox.swift @@ -0,0 +1,172 @@ +import Foundation + +/// What the HOST allows a widget to reach, independent of what the widget's `_meta.ui.csp` asks +/// for. A server's CSP declaration is a request: the effective CSP is the declaration +/// INTERSECTED with this policy, so a server update can never widen what an employee's app lets +/// a widget contact. The default allows nothing. +public struct MCPAppsSandboxPolicy: Sendable, Equatable { + /// Origins a widget may `fetch`/WebSocket to. Exact `https://host[:port]` / `wss://host[:port]`, + /// or a `https://*.example.com` wildcard. A declared origin is granted only if it matches an + /// entry here. + public var allowedConnectOrigins: [String] + /// Origins a widget may load scripts, styles, images, fonts and media from. + public var allowedResourceOrigins: [String] + /// Origins a widget may embed in a frame. + public var allowedFrameOrigins: [String] + /// Adds `allow-same-origin` to the widget iframe's sandbox. Off by default: the widget then + /// runs in an opaque origin with no access to storage or cookies. Turn on only for a widget + /// that needs `localStorage` and is otherwise trusted. + public var allowsSameOrigin: Bool + /// Makes the WKWebView inspectable in Safari's Develop menu. Debug builds only. + public var isInspectable: Bool + + public init( + allowedConnectOrigins: [String] = [], allowedResourceOrigins: [String] = [], + allowedFrameOrigins: [String] = [], allowsSameOrigin: Bool = false, isInspectable: Bool = false + ) { + self.allowedConnectOrigins = allowedConnectOrigins + self.allowedResourceOrigins = allowedResourceOrigins + self.allowedFrameOrigins = allowedFrameOrigins + self.allowsSameOrigin = allowsSameOrigin + self.isInspectable = isInspectable + } + + /// Allow no network, no frames, no storage. + public static let closed = MCPAppsSandboxPolicy() +} + +/// Pure functions behind the web-view sandbox: origin validation, CSP intersection and header +/// construction, and the WebKit content-rule list that blocks network loads as a second layer +/// beneath the CSP. No WebKit types, so all of it is unit-tested directly. +public enum MCPAppsSandbox { + // MARK: origins + + /// A server-supplied origin is interpolated into a CSP header, so it is validated strictly + /// (a `;` or space would let a hostile server inject its own directives). Accepts only + /// `https://` / `wss://` + a DNS-style host (optionally `*.`-prefixed) + optional port. + public static func isSafeOrigin(_ origin: String) -> Bool { + parse(origin) != nil + } + + struct ParsedOrigin: Equatable { + var scheme: String + var host: String // lowercased; may start with "*." + var port: Int? + var isWildcard: Bool { host.hasPrefix("*.") } + } + + static func parse(_ origin: String) -> ParsedOrigin? { + guard origin.count <= 253, origin.utf8.allSatisfy({ $0 < 0x80 }) else { return nil } + let lower = origin.lowercased() + let schemes = ["https://": "https", "wss://": "wss"] + guard let (prefix, scheme) = schemes.first(where: { lower.hasPrefix($0.key) }) else { return nil } + var rest = String(lower.dropFirst(prefix.count)) + var port: Int? + if let colon = rest.lastIndex(of: ":") { + let portString = String(rest[rest.index(after: colon)...]) + guard !portString.isEmpty, portString.count <= 5, portString.allSatisfy(\.isNumber), + let p = Int(portString), (1...65535).contains(p) else { return nil } + port = p + rest = String(rest[..= 2, !host.isEmpty else { return nil } // no bare TLDs / single labels + for label in labels { + guard !label.isEmpty, label.count <= 63, !label.hasPrefix("-"), !label.hasSuffix("-"), + label.allSatisfy({ $0.isASCII && ($0.isLetter || $0.isNumber || $0 == "-") }) else { return nil } + } + return ParsedOrigin(scheme: scheme, host: (wildcard ? "*." : "") + host, port: port) + } + + /// True when `declared` is granted by `pattern`. Exact origin match, or `pattern` is a + /// `*.parent` wildcard covering a concrete subdomain of `parent` (same scheme and port). + /// A declared wildcard is granted only by an identical wildcard entry. + static func matches(declared: ParsedOrigin, pattern: ParsedOrigin) -> Bool { + guard declared.scheme == pattern.scheme, declared.port == pattern.port else { return false } + if declared.host == pattern.host { return true } + guard pattern.isWildcard, !declared.isWildcard else { return false } + return declared.host.hasSuffix(String(pattern.host.dropFirst(1))) // ".example.com" + } + + private static func intersect(_ declared: [String], _ allowed: [String]) -> [String] { + let patterns = allowed.compactMap(parse) + var seen = Set() + var result: [String] = [] + for origin in declared { + guard let parsed = parse(origin), patterns.contains(where: { matches(declared: parsed, pattern: $0) }) else { continue } + let normalized = canonical(parsed) + if seen.insert(normalized).inserted { result.append(normalized) } + } + return result + } + + private static func canonical(_ o: ParsedOrigin) -> String { + "\(o.scheme)://\(o.host)" + (o.port.map { ":\($0)" } ?? "") + } + + // MARK: CSP + + /// The CSP the host will actually enforce: each declared origin list intersected with the + /// host's allowlist. `baseUriDomains` are never granted (a widget has no business rebasing + /// URLs). + public static func effectiveCSP(declared: MCPAppCSP, policy: MCPAppsSandboxPolicy) -> MCPAppCSP { + MCPAppCSP( + connectDomains: intersect(declared.connectDomains, policy.allowedConnectOrigins), + resourceDomains: intersect(declared.resourceDomains, policy.allowedResourceOrigins), + frameDomains: intersect(declared.frameDomains, policy.allowedFrameOrigins), + baseUriDomains: []) + } + + /// The `Content-Security-Policy` header value for a widget document. Every origin in `csp` + /// must already have passed `effectiveCSP` (each is re-validated here regardless). + public static func headerValue(for csp: MCPAppCSP) -> String { + func sources(_ base: [String], _ extra: [String]) -> String { + (base + extra.filter(isSafeOrigin)).joined(separator: " ") + } + let frames = csp.frameDomains.filter(isSafeOrigin) + return [ + "default-src 'none'", + "script-src \(sources(["'self'", "'unsafe-inline'"], csp.resourceDomains))", + "style-src \(sources(["'self'", "'unsafe-inline'"], csp.resourceDomains))", + "img-src \(sources(["'self'", "data:", "blob:"], csp.resourceDomains))", + "font-src \(sources(["'self'", "data:"], csp.resourceDomains))", + "media-src \(sources(["'self'", "data:", "blob:"], csp.resourceDomains))", + "connect-src \(csp.connectDomains.filter(isSafeOrigin).isEmpty ? "'none'" : csp.connectDomains.filter(isSafeOrigin).joined(separator: " "))", + "frame-src \(frames.isEmpty ? "'none'" : frames.joined(separator: " "))", + "base-uri 'none'", + "form-action 'none'", + "object-src 'none'", + ].joined(separator: "; ") + } + + // MARK: WebKit content rules (second layer under the CSP) + + /// A WKContentRuleList (JSON) that blocks every http(s)/ws(s)/ftp load except the effective + /// allowed origins. It applies to the whole web view, so it holds even if a CSP were ever + /// mis-served. WebKit's content-blocker regex has no groups or alternation, hence one rule per + /// scheme and per allowed origin. + public static func contentRuleListJSON(for csp: MCPAppCSP) -> String { + var rules: [[String: Any]] = ["^https?://", "^wss?://", "^ftp://"].map { + ["trigger": ["url-filter": $0], "action": ["type": "block"]] + } + let allowed = Set(csp.connectDomains + csp.resourceDomains + csp.frameDomains).compactMap(parse) + for origin in allowed.sorted(by: { canonical($0) < canonical($1) }) { + let host = origin.isWildcard + ? "[a-z0-9.-]+\\." + escapeForRegex(String(origin.host.dropFirst(2))) + : escapeForRegex(origin.host) + // Content-blocker regex has no groups, so an explicit default port (`:443`) in a URL + // is not matched for a portless origin — WebKit normalizes it away in practice. + let filter = "^\(origin.scheme)://\(host)" + (origin.port.map { ":\($0)" } ?? "") + "/" + rules.append(["trigger": ["url-filter": filter], "action": ["type": "ignore-previous-rules"]]) + } + let data = (try? JSONSerialization.data(withJSONObject: rules, options: [.sortedKeys])) ?? Data("[]".utf8) + return String(decoding: data, as: UTF8.self) + } + + private static func escapeForRegex(_ s: String) -> String { + s.replacingOccurrences(of: ".", with: "\\.") + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSessionBackend.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSessionBackend.swift new file mode 100644 index 0000000..6085bb0 --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPAppsSessionBackend.swift @@ -0,0 +1,81 @@ +import Foundation +import LocalLMLabSDKCore + +// For hosts with a model in the loop: route a +// widget's calls and messages through the LocalLMLabSession, so they pass the session's +// ToolCallAuthorizer (which sees `initiator: .app(instance:)`) and land in its HostTranscript. +// Model-less hosts (a widget browser, say) keep using MCPServerManagerBackend and the bridge policy. + +/// An `MCPAppsBackend` whose `tools/call` goes through `session.callMCPTool(…, initiator: .app)`. +/// +/// The bridge still enforces its own rules first (the tool must be app-callable, and +/// `MCPAppsBridgePolicy.authorizeToolCall` must allow it; the default policy does); then the +/// session's authorizer decides. Resource reads go to the server directly. +@MainActor +public struct MCPAppsSessionBackend: MCPAppsBackend { + public let session: LocalLMLabSession + public let manager: MCPServerManager + public let server: MCPServerID + /// Identifies this widget in records, the authorizer and `HostTranscript.appOnlyCallCount`. + /// Conventionally the id of the tool-call record the widget was opened from. + public let instance: String + + public init(session: LocalLMLabSession, manager: MCPServerManager, server: MCPServerID, instance: String) { + self.session = session + self.manager = manager + self.server = server + self.instance = instance + } + + public func callTool(name: String, arguments: [String: MCPValue]) async -> Result { + await session.callMCPTool(server: server, tool: name, arguments: arguments, initiator: .app(instance: instance)) + } + + public func readResource(uri: String) async -> Result { + await manager.readResource(server: server, uri: uri) + } +} + +extension MCPAppsHostActions { + /// Actions that send a widget's `ui/message` into `session` as a user turn and its + /// `ui/update-model-context` as model context — **only reached when the bridge policy allows + /// them** (`authorizeMessage` / `authorizeModelContextUpdate` deny by default). + /// + /// - Parameters: + /// - onReply: called with the model's reply to a widget-sent message (or the error). + public static func session( + _ session: LocalLMLabSession, + instance: String, + openLink: @escaping @MainActor (URL) -> Void = { _ in }, + onReply: @escaping @MainActor (Result) -> Void = { _ in }, + requestDisplayMode: @escaping @MainActor (_ mode: String) -> Bool = { _ in false } + ) -> MCPAppsHostActions { + MCPAppsHostActions( + openLink: openLink, + sendMessage: { text in + Task { @MainActor in + do { onReply(.success(try await session.respond(to: text, fromAppInstance: instance))) } catch { onReply(.failure(error)) } + } + }, + updateModelContext: { content, structured in + session.updateModelContext(contextText(content: content, structured: structured), fromAppInstance: instance) + }, + requestDisplayMode: requestDisplayMode) + } + + /// The text a model gets for `ui/update-model-context`: the text items, then the structured + /// content as JSON. + static func contextText(content: [MCPValue], structured: MCPValue?) -> String { + var parts: [String] = content.compactMap { item in + guard case .object(let object) = item else { return nil } + if case .string(let text)? = object["text"] { return text } + return nil + } + if let structured { + let encoder = JSONEncoder() + encoder.outputFormatting = [.sortedKeys] + if let data = try? encoder.encode(structured) { parts.append(String(decoding: data, as: UTF8.self)) } + } + return parts.joined(separator: "\n") + } +} diff --git a/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPToolInputForm.swift b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPToolInputForm.swift new file mode 100644 index 0000000..6cd57dc --- /dev/null +++ b/MCPAppsHost/Sources/LocalLMLabSDKMCPAppsHost/MCPToolInputForm.swift @@ -0,0 +1,171 @@ +import Foundation +import LocalLMLabSDKCore + +/// One input a tool takes, derived from its JSON Schema, in the terms a form needs. +public struct MCPToolInputField: Identifiable, Sendable, Equatable { + public enum Kind: Sendable, Equatable { + case string + case integer + case number + case boolean + /// A string restricted to these values. + case choice([String]) + /// A list of simple values, entered comma-separated. `integer`/`number`/`string` elements. + case list(element: ListElement) + /// Anything the form can't model (objects, unions, nested arrays): raw JSON text. + case json + } + + public enum ListElement: Sendable, Equatable { case string, integer, number } + + public var id: String { name } + public let name: String + public let summary: String + public let isRequired: Bool + public let kind: Kind + /// The schema default, rendered as text for prefilling the form. + public let defaultText: String? +} + +public enum MCPToolInputError: Error, Equatable, CustomStringConvertible { + case missingRequired(String) + case invalid(field: String, reason: String) + + public var description: String { + switch self { + case .missingRequired(let name): return "\(name) is required" + case .invalid(let field, let reason): return "\(field): \(reason)" + } + } +} + +/// Turns a tool's `inputSchema` into form fields, and typed user input back into tool arguments. +/// Pure logic: no UI. Tolerant of real-world schemas — anything it can't model becomes a JSON +/// field, and the SERVER still validates (the form is a convenience, never a security check). +public struct MCPToolInputForm: Sendable, Equatable { + public let fields: [MCPToolInputField] + + /// True when there is nothing to ask the user: the tool takes no inputs. + public var isEmpty: Bool { fields.isEmpty } + /// True when at least one input is mandatory (the tool can't be launched blank). + public var hasRequiredFields: Bool { fields.contains { $0.isRequired } } + + public init(schema: Data) { + guard let root = (try? JSONSerialization.jsonObject(with: schema)) as? [String: Any], + let properties = root["properties"] as? [String: Any] else { + fields = [] + return + } + let required = Set((root["required"] as? [Any])?.compactMap { $0 as? String } ?? []) + fields = properties.compactMap { name, raw -> MCPToolInputField? in + guard let prop = raw as? [String: Any] else { return nil } + return Self.field(name: name, prop: prop, required: required.contains(name)) + } + .sorted { ($0.isRequired ? 0 : 1, $0.name) < ($1.isRequired ? 0 : 1, $1.name) } + } + + private static func field(name: String, prop: [String: Any], required: Bool) -> MCPToolInputField { + var summary = (prop["description"] as? String) ?? (prop["title"] as? String) ?? "" + if summary.count > 400 { summary = String(summary.prefix(400)) + "…" } + let kind = Self.kind(of: prop) + var defaultText: String? + if let d = prop["default"], !(d is NSNull) { + switch d { + case let s as String: defaultText = s + case let n as NSNumber: defaultText = CFGetTypeID(n) == CFBooleanGetTypeID() ? (n.boolValue ? "true" : "false") : "\(n)" + default: + if let data = try? JSONSerialization.data(withJSONObject: d, options: [.fragmentsAllowed]) { defaultText = String(decoding: data, as: UTF8.self) } + } + } + return MCPToolInputField(name: name, summary: summary, isRequired: required, kind: kind, defaultText: defaultText) + } + + private static func kind(of prop: [String: Any]) -> MCPToolInputField.Kind { + if let options = prop["enum"] as? [Any] { + let strings = options.compactMap { $0 as? String } + return strings.count == options.count && !strings.isEmpty ? .choice(strings) : .json + } + // `type` may be a string or a list like ["string", "null"]: take the single non-null type. + var type = prop["type"] as? String + if let types = prop["type"] as? [Any] { + let real = types.compactMap { $0 as? String }.filter { $0 != "null" } + type = real.count == 1 ? real[0] : nil + } + switch type { + case "string": return .string + case "integer": return .integer + case "number": return .number + case "boolean": return .boolean + case "array": + switch ((prop["items"] as? [String: Any])?["type"] as? String) { + case "string": return .list(element: .string) + case "integer": return .list(element: .integer) + case "number": return .list(element: .number) + default: return .json + } + default: return .json + } + } + + /// Builds the tool arguments from what the user typed (`values` maps field name to text; + /// booleans are `"true"`/`"false"`). Blank optional fields are omitted; a blank required field, + /// or text that doesn't fit the field's type, is an error. + public func arguments(from values: [String: String]) -> Result<[String: MCPValue], MCPToolInputError> { + var out: [String: MCPValue] = [:] + for field in fields { + let raw = (values[field.name] ?? "").trimmingCharacters(in: .whitespacesAndNewlines) + if raw.isEmpty { + if field.isRequired && field.kind != .boolean { return .failure(.missingRequired(field.name)) } + if field.kind == .boolean, values[field.name] != nil { out[field.name] = .bool(false) } + continue + } + switch Self.convert(raw, kind: field.kind) { + case .success(let value): out[field.name] = value + case .failure(let reason): return .failure(.invalid(field: field.name, reason: reason)) + } + } + return .success(out) + } + + private enum Conversion { case success(MCPValue), failure(String) } + + private static func convert(_ raw: String, kind: MCPToolInputField.Kind) -> Conversion { + switch kind { + case .string: return .success(.string(raw)) + case .integer: + guard let n = Int(raw) else { return .failure("must be a whole number") } + return .success(.number(Double(n))) + case .number: + guard let n = Double(raw), n.isFinite else { return .failure("must be a number") } + return .success(.number(n)) + case .boolean: + switch raw.lowercased() { + case "true", "yes", "1": return .success(.bool(true)) + case "false", "no", "0": return .success(.bool(false)) + default: return .failure("must be true or false") + } + case .choice(let options): + return options.contains(raw) ? .success(.string(raw)) : .failure("must be one of \(options.joined(separator: ", "))") + case .list(let element): + let parts = raw.split(separator: ",").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty } + var items: [MCPValue] = [] + for part in parts { + switch element { + case .string: items.append(.string(part)) + case .integer: + guard let n = Int(part) else { return .failure("\"\(part)\" is not a whole number") } + items.append(.number(Double(n))) + case .number: + guard let n = Double(part), n.isFinite else { return .failure("\"\(part)\" is not a number") } + items.append(.number(n)) + } + } + return .success(.array(items)) + case .json: + guard let data = raw.data(using: .utf8), let value = try? JSONDecoder().decode(MCPValue.self, from: data) else { + return .failure("is not valid JSON") + } + return .success(value) + } + } +} diff --git a/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppLifecycleTests.swift b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppLifecycleTests.swift new file mode 100644 index 0000000..2631197 --- /dev/null +++ b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppLifecycleTests.swift @@ -0,0 +1,235 @@ +import AppKit +import Foundation +import FoundationModels +import LocalLMLabSDKCore +import Testing + +import LocalLMLabSDKMCPAppsHost + +// Widget lifecycle: content-addressed cache, re-creation decisions, revalidation, and the +// live-widget limit. Public API only. + +private let server = MCPServerID(rawValue: "https://example.invalid/mcp") +private let uri = "ui://example/list" + +private func widget(_ marker: String) -> MCPResourceContent { + MCPResourceContent(uri: uri, mimeType: "text/html;profile=mcp-app", + text: "

\(marker)

", + meta: .object(["ui": .object(["prefersBorder": .bool(true)])])) +} + +private func tool(outputSchema: String = #"{"type":"object","properties":{"tasks":{"type":"array"}}}"#, + readOnly: Bool? = nil) -> MCPToolDescriptor { + MCPToolDescriptor(serverID: server, name: "find", description: "", rawSchema: Data("{}".utf8), estimatedTokens: 1, + outputSchema: Data(outputSchema.utf8), + meta: .object(["ui": .object(["resourceUri": .string(uri)])]), + annotations: readOnly.map { MCPToolAnnotations(readOnlyHint: $0) }) +} + +@MainActor +private struct FakeSource: MCPAppWidgetSource { + var current: MCPResourceContent? + var descriptor: MCPToolDescriptor? + func currentResource(uri: String) async -> Result { + current.map { .success($0) } ?? .failure(.unreachable) + } + func currentDescriptor(tool: String) -> MCPToolDescriptor? { descriptor } +} + +@MainActor +private func freshCache(maxBytes: Int = 64 * 1024 * 1024) -> MCPAppWidgetCache { + MCPAppWidgetCache(directory: FileManager.default.temporaryDirectory.appendingPathComponent("MCPAppWidgetCacheTests-\(UUID().uuidString)"), + maxBytes: maxBytes) +} + +@MainActor +@Suite(.serialized) +struct MCPAppLifecycleTests { + // MARK: decision table + + @Test func decisions() { + typealias R = MCPAppRecreation + // reachable, same version + #expect(R.decide(cachedSHA256: "a", currentSHA256: "a", outputSchemaUnchanged: true, toolIsReadOnly: false) == .render(viewOnly: false)) + // same version, schema changed: the cached widget matches the stored result + #expect(R.decide(cachedSHA256: "a", currentSHA256: "a", outputSchemaUnchanged: false, toolIsReadOnly: false) == .render(viewOnly: false)) + // newer version, same schema: current widget with stored result + #expect(R.decide(cachedSHA256: "a", currentSHA256: "b", outputSchemaUnchanged: true, toolIsReadOnly: false) == .render(viewOnly: false)) + // newer version, schema changed: refresh, automatic only for read-only tools + #expect(R.decide(cachedSHA256: "a", currentSHA256: "b", outputSchemaUnchanged: false, toolIsReadOnly: true) == .needsRefresh(mayRefreshAutomatically: true)) + #expect(R.decide(cachedSHA256: "a", currentSHA256: "b", outputSchemaUnchanged: false, toolIsReadOnly: false) == .needsRefresh(mayRefreshAutomatically: false)) + // unknown cached version counts as changed + #expect(R.decide(cachedSHA256: nil, currentSHA256: "b", outputSchemaUnchanged: false, toolIsReadOnly: false) == .needsRefresh(mayRefreshAutomatically: false)) + #expect(R.decide(cachedSHA256: nil, currentSHA256: "b", outputSchemaUnchanged: true, toolIsReadOnly: false) == .render(viewOnly: false)) + // offline + #expect(R.decide(cachedSHA256: "a", currentSHA256: nil, outputSchemaUnchanged: true, toolIsReadOnly: false) == .render(viewOnly: true)) + #expect(R.decide(cachedSHA256: nil, currentSHA256: nil, outputSchemaUnchanged: true, toolIsReadOnly: false) == .unavailable) + // pins + #expect(R.decide(cachedSHA256: "a", currentSHA256: "b", outputSchemaUnchanged: true, toolIsReadOnly: false, approvedSHA256: ["a"]) == .notApproved) + #expect(R.decide(cachedSHA256: "a", currentSHA256: "b", outputSchemaUnchanged: true, toolIsReadOnly: false, approvedSHA256: ["b"]) == .render(viewOnly: false)) + #expect(R.decide(cachedSHA256: "a", currentSHA256: nil, outputSchemaUnchanged: true, toolIsReadOnly: false, approvedSHA256: ["b"]) == .notApproved) + } + + // MARK: cache + + @Test func theCacheKeepsVersionsAndRemembersWhichCallUsedWhich() throws { + let cache = freshCache() + let stored = try cache.store(widget("v1"), requestedURI: uri) + cache.bind(recordID: "call-1", sha256: stored.sha256) + let back = try #require(cache.resource(sha256: stored.sha256)) + #expect(back == stored) + #expect(back.info.prefersBorder == true) + #expect(cache.sha256(forRecord: "call-1") == stored.sha256) + + // A second instance on the same directory sees the same versions and bindings. + let reopened = MCPAppWidgetCache(directory: cache.directory) + #expect(reopened.sha256(forRecord: "call-1") == stored.sha256) + #expect(reopened.resource(sha256: stored.sha256) == stored) + #expect(MCPAppRecreation.cachedResource(recordID: "call-1", cache: reopened) == stored) + } + + @Test func theCacheEvictsTheLeastRecentlyUsedVersion() throws { + let cache = freshCache() + let v1 = try cache.store(widget("v1"), requestedURI: uri) + cache.maxBytes = cache.storedBytes * 2 + cache.storedBytes / 2 // room for two versions + let v2 = try cache.store(widget("v2"), requestedURI: uri) + _ = cache.resource(sha256: v1.sha256) // v1 is now the most recently used + let v3 = try cache.store(widget("v3"), requestedURI: uri) + #expect(cache.storedVersions == [v1.sha256, v3.sha256]) + #expect(cache.resource(sha256: v2.sha256) == nil) + } + + @Test func aTamperedCacheFileIsNotServed() throws { + let cache = freshCache() + let v1 = try cache.store(widget("v1"), requestedURI: uri) + let other = try JSONEncoder().encode(["requestedURI": uri]) + try other.write(to: cache.directory.appendingPathComponent("\(v1.sha256).json")) + #expect(cache.resource(sha256: v1.sha256) == nil) + } + + // MARK: revalidation + + @Test func revalidateRendersTheCurrentWidgetWhenItIsTheSame() async throws { + let cache = freshCache() + let v1 = try cache.store(widget("v1"), requestedURI: uri) + cache.bind(recordID: "c", sha256: v1.sha256) + let r = await MCPAppRecreation.revalidate(recordID: "c", resourceURI: uri, callDescriptor: tool(), + source: FakeSource(current: widget("v1"), descriptor: tool()), cache: cache) + #expect(r.decision == .render(viewOnly: false)) + #expect(r.resource == v1) + #expect(!r.widgetChanged) + } + + @Test func revalidateMovesACallToANewerCompatibleWidget() async throws { + let cache = freshCache() + let v1 = try cache.store(widget("v1"), requestedURI: uri) + cache.bind(recordID: "c", sha256: v1.sha256) + let r = await MCPAppRecreation.revalidate(recordID: "c", resourceURI: uri, callDescriptor: tool(), + source: FakeSource(current: widget("v2"), descriptor: tool()), cache: cache) + #expect(r.decision == .render(viewOnly: false)) + #expect(r.widgetChanged) + let v2 = try #require(r.resource) + #expect(v2.sha256 != v1.sha256) + #expect(cache.sha256(forRecord: "c") == v2.sha256) // re-bound to the version now shown + } + + @Test func revalidateAsksForARefreshWhenTheOutputChangedShape() async throws { + let cache = freshCache() + let v1 = try cache.store(widget("v1"), requestedURI: uri) + cache.bind(recordID: "c", sha256: v1.sha256) + let newShape = tool(outputSchema: #"{"type":"object","properties":{"items":{"type":"array"}}}"#, readOnly: true) + let r = await MCPAppRecreation.revalidate(recordID: "c", resourceURI: uri, callDescriptor: tool(), + source: FakeSource(current: widget("v2"), descriptor: newShape), cache: cache) + #expect(r.decision == .needsRefresh(mayRefreshAutomatically: true)) + #expect(r.resource == nil) + #expect(cache.sha256(forRecord: "c") == v1.sha256) // not re-bound + } + + @Test func revalidateOfflineShowsTheCachedVersionViewOnly() async throws { + let cache = freshCache() + let v1 = try cache.store(widget("v1"), requestedURI: uri) + cache.bind(recordID: "c", sha256: v1.sha256) + let offline = await MCPAppRecreation.revalidate(recordID: "c", resourceURI: uri, callDescriptor: tool(), + source: FakeSource(current: nil, descriptor: nil), cache: cache) + #expect(offline.decision == .render(viewOnly: true)) + #expect(offline.resource == v1) + let nothing = await MCPAppRecreation.revalidate(recordID: "unknown", resourceURI: uri, callDescriptor: tool(), + source: FakeSource(current: nil, descriptor: nil), cache: cache) + #expect(nothing.decision == .unavailable) + } + + @Test func revalidateHonorsAPin() async throws { + let cache = freshCache() + let v1 = try cache.store(widget("v1"), requestedURI: uri) + cache.bind(recordID: "c", sha256: v1.sha256) + let r = await MCPAppRecreation.revalidate(recordID: "c", resourceURI: uri, callDescriptor: tool(), + source: FakeSource(current: widget("v2"), descriptor: tool()), cache: cache, + approvedSHA256: [v1.sha256]) + #expect(r.decision == .notApproved) + #expect(r.resource == nil) + } + + @Test func toolInputComesFromTheCallsArguments() throws { + let args = try GeneratedContent(json: #"{"startDate":"today","limit":5}"#) + #expect(MCPAppRecreation.toolInput(from: args) == ["startDate": .string("today"), "limit": .number(5)]) + #expect(MCPAppRecreation.toolInput(from: nil).isEmpty) + } + + // MARK: live-widget limit + + private func controller() throws -> MCPAppViewController { + let resource = try MCPAppResource(content: widget("pool"), requestedURI: uri) + return MCPAppViewController(resource: resource, tools: [], backend: NoBackend(), + configuration: MCPAppsBridgeConfiguration(hostName: "T", hostVersion: "1", + teardownTimeout: .milliseconds(50))) + } + + @Test func thePoolKeepsTheLimitAndSnapshotsWhatItReleases() async throws { + _ = NSApplication.shared + let pool = MCPAppViewPool(limit: 2) + var made = 0 + var windows: [NSWindow] = [] + func open(_ key: String) async throws -> MCPAppViewController { + let c = try pool.controller(for: key) { made += 1; return try controller() } + if c.webView.window == nil { + let w = NSWindow(contentRect: NSRect(x: 0, y: 0, width: 300, height: 200), styleMask: [.titled], backing: .buffered, defer: false) + w.contentView = c.webView + w.orderFrontRegardless() + windows.append(w) + await c.load() + try await Task.sleep(for: .milliseconds(400)) + } + return c + } + let first = try await open("k1") + _ = try await open("k2") + _ = try await open("k3") + #expect(pool.liveKeys == ["k2", "k3"]) + #expect(!pool.isLive("k1")) + + // The released widget is closed and pictured. + let deadline = ContinuousClock.now + .seconds(5) + while pool.snapshot(for: "k1") == nil || first.bridge.state != .tornDown, ContinuousClock.now < deadline { + try await Task.sleep(for: .milliseconds(50)) + } + #expect(pool.snapshot(for: "k1") != nil) + #expect(first.bridge.state == .tornDown) + + // Asking again re-creates it (and releases the least recent of the others). + _ = try await open("k1") + #expect(made == 4) + #expect(pool.liveKeys == ["k3", "k1"]) + + pool.limit = 1 + #expect(pool.liveKeys == ["k1"]) + await pool.releaseAll() + #expect(pool.liveKeys.isEmpty) + for w in windows { w.contentView = nil; w.orderOut(nil) } + } +} + +@MainActor +private struct NoBackend: MCPAppsBackend { + func callTool(name: String, arguments: [String: MCPValue]) async -> Result { .failure(.unreachable) } + func readResource(uri: String) async -> Result { .failure(.unreachable) } +} diff --git a/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppMetadataTests.swift b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppMetadataTests.swift new file mode 100644 index 0000000..4545ca6 --- /dev/null +++ b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppMetadataTests.swift @@ -0,0 +1,109 @@ +import Foundation +import LocalLMLabSDKCore +import Testing + +import LocalLMLabSDKMCPAppsHost + +// Public API only (no @testable), like Components' tests. + +private func tool(_ name: String, meta: MCPValue?) -> MCPToolDescriptor { + MCPToolDescriptor( + serverID: MCPServerID(rawValue: "https://example.invalid/mcp"), name: name, description: "", + rawSchema: Data("{}".utf8), estimatedTokens: 0, meta: meta) +} + +@Suite struct MCPAppMetadataTests { + // Tool-side parsing (resource URI, visibility) is tested in Core: MCPAppLinkTests. + + @Test func resourceInfoParsesTodoistShapedEmptyCSP() { + let meta: MCPValue = .object([ + "ui": .object([ + "csp": .object(["connectDomains": .array([]), "resourceDomains": .array([])]), + "prefersBorder": .bool(true), + ]), + "openai/widgetDomain": .string("https://ai.todoist.net"), + ]) + let info = MCPAppResourceInfo(meta: meta) + #expect(info.csp.isEmpty) + #expect(info.permissions.isEmpty) + #expect(info.prefersBorder == true) + } + + @Test func resourceInfoParsesDomainsAndPermissions() { + let meta: MCPValue = .object(["ui": .object([ + "csp": .object([ + "connectDomains": .array([.string("https://api.example.com")]), + "resourceDomains": .array([.string("https://cdn.example.com"), .number(3)]), + ]), + "permissions": .object(["camera": .object([:]), "microphone": .bool(false)]), + "domain": .string("abc.example"), + ])]) + let info = MCPAppResourceInfo(meta: meta) + #expect(info.csp.connectDomains == ["https://api.example.com"]) + #expect(info.csp.resourceDomains == ["https://cdn.example.com"]) // non-strings dropped + #expect(info.permissions == MCPAppPermissions(camera: true)) + #expect(info.domain == "abc.example") + } + + @Test func malformedMetaYieldsMostRestrictiveInfo() { + let info = MCPAppResourceInfo(meta: .string("nope")) + #expect(info == MCPAppResourceInfo()) + #expect(MCPAppResourceInfo(meta: nil) == MCPAppResourceInfo()) + } + + @Test func resourceWrapsHTMLAndHashesIt() throws { + let content = MCPResourceContent( + uri: "ui://x/a", mimeType: "text/html;profile=mcp-app", text: "hi", + meta: .object(["ui": .object(["prefersBorder": .bool(true)])])) + let resource = try MCPAppResource(content: content, requestedURI: "ui://x/a") + #expect(resource.html == "hi") + #expect(resource.info.prefersBorder == true) + // sha256("hi") + #expect(resource.sha256.count == 64) + #expect(resource.sha256 == (try MCPAppResource(content: content, requestedURI: "ui://x/a")).sha256) + let other = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html;profile=mcp-app", text: "ho") + #expect(try MCPAppResource(content: other, requestedURI: "ui://x/a").sha256 != resource.sha256) + } + + @Test func resourceKnownHashVector() throws { + // SHA-256("abc"), a standard test vector. + let content = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html;profile=mcp-app", text: "abc") + #expect(try MCPAppResource(content: content, requestedURI: "ui://x/a").sha256 + == "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad") + } + + @Test func resourceUsesRequestedURINotEchoedOne() throws { + let content = MCPResourceContent(uri: "ui://other/thing", mimeType: "text/html;profile=mcp-app", text: "x") + #expect(try MCPAppResource(content: content, requestedURI: "ui://x/a").uri == "ui://x/a") + } + + @Test func resourceContentMetaWinsOverListedMeta() throws { + let read: MCPValue = .object(["ui": .object(["prefersBorder": .bool(false)])]) + let listed: MCPValue = .object(["ui": .object(["prefersBorder": .bool(true)])]) + let content = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html;profile=mcp-app", text: "x", meta: read) + #expect(try MCPAppResource(content: content, requestedURI: "ui://x/a", listedMeta: listed).info.prefersBorder == false) + let bare = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html;profile=mcp-app", text: "x") + #expect(try MCPAppResource(content: bare, requestedURI: "ui://x/a", listedMeta: listed).info.prefersBorder == true) + } + + @Test func resourceRejectsBadInputs() { + let good = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html;profile=mcp-app", text: "x") + #expect(throws: MCPAppResourceError.notAUIResource("https://x/a")) { + try MCPAppResource(content: good, requestedURI: "https://x/a") + } + let plainHTML = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html", text: "x") + #expect(throws: MCPAppResourceError.wrongMimeType("text/html")) { + try MCPAppResource(content: plainHTML, requestedURI: "ui://x/a") + } + let empty = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html;profile=mcp-app") + #expect(throws: MCPAppResourceError.noBody) { try MCPAppResource(content: empty, requestedURI: "ui://x/a") } + let badBlob = MCPResourceContent(uri: "ui://x/a", mimeType: "text/html;profile=mcp-app", blob: "!!!not base64!!!") + #expect(throws: MCPAppResourceError.invalidBase64) { try MCPAppResource(content: badBlob, requestedURI: "ui://x/a") } + } + + @Test func resourceAcceptsBase64BlobAndMimeWhitespace() throws { + let blob = Data("hi".utf8).base64EncodedString() + let content = MCPResourceContent(uri: "ui://x/a", mimeType: "Text/HTML; profile=mcp-app", blob: blob) + #expect(try MCPAppResource(content: content, requestedURI: "ui://x/a").html == "hi") + } +} diff --git a/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppViewControllerTests.swift b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppViewControllerTests.swift new file mode 100644 index 0000000..b9d5458 --- /dev/null +++ b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppViewControllerTests.swift @@ -0,0 +1,225 @@ +import AppKit +import Foundation +import LocalLMLabSDKCore +import Testing +import WebKit + +import LocalLMLabSDKMCPAppsHost + +// Loads a REAL WKWebView with a hostile probe widget and checks, from the host side, what the +// containment layers let through. The widget reports over the normal bridge (tools/call +// "report") so the test needs no JS-evaluation hooks. + +@MainActor +private final class Recorder { + var calls: [(name: String, arguments: [String: MCPValue])] = [] + var audit: [MCPAppsAuditEvent] = [] + var openedLinks: [URL] = [] +} + +@MainActor +private struct Backend: MCPAppsBackend { + let recorder: Recorder + func callTool(name: String, arguments: [String: MCPValue]) async -> Result { + recorder.calls.append((name, arguments)) + return .success(MCPToolResult(text: "ok", structuredContent: .object(["echo": .string(name)]))) + } + func readResource(uri: String) async -> Result { .failure(.notConnected) } +} + +private func tool(_ name: String) -> MCPToolDescriptor { + MCPToolDescriptor(serverID: MCPServerID(rawValue: "https://example.invalid/mcp"), name: name, description: "", rawSchema: Data("{}".utf8), estimatedTokens: 0) +} + +/// A widget that speaks the MCP Apps handshake through `window.parent.postMessage` (as the +/// bundled ext-apps client does) and then probes its own containment. +private let probeWidget = #""" + +"""# + +/// A widget that reports the host context it is given and asks for fullscreen when told to. +private let contextWidget = #""" + +"""# + +@MainActor +@Suite(.serialized) struct MCPAppViewControllerTests { + private func makeController(recorder: Recorder, html: String = probeWidget, meta: MCPValue? = nil, sandbox: MCPAppsSandboxPolicy = .closed, modes: [String] = ["inline"]) throws -> MCPAppViewController { + _ = NSApplication.shared + let content = MCPResourceContent(uri: "ui://probe/w", mimeType: "text/html;profile=mcp-app", text: html, meta: meta) + let resource = try MCPAppResource(content: content, requestedURI: "ui://probe/w") + return MCPAppViewController( + resource: resource, tools: [tool("report")], backend: Backend(recorder: recorder), + configuration: MCPAppsBridgeConfiguration(hostName: "TestHost", hostVersion: "1", teardownTimeout: .milliseconds(200)), + sandboxPolicy: sandbox, + actions: MCPAppsHostActions(openLink: { recorder.openedLinks.append($0) }), + supportedDisplayModes: modes, + audit: { recorder.audit.append($0) }) + } + + private func waitForReport(_ recorder: Recorder, key: String = "done", timeout: Duration = .seconds(15)) async -> Bool { + let deadline = ContinuousClock.now + timeout + while ContinuousClock.now < deadline { + if recorder.calls.contains(where: { $0.arguments["key"] == .string(key) }) { return true } + try? await Task.sleep(for: .milliseconds(50)) + } + return false + } + + private func reports(_ recorder: Recorder) -> [String: String] { + var out: [String: String] = [:] + for call in recorder.calls where call.name == "report" { + if case .string(let k)? = call.arguments["key"], case .string(let v)? = call.arguments["value"] { out[k] = v } + } + return out + } + + @Test func aHostileWidgetIsContainedAndTheBridgeWorksEndToEnd() async throws { + let recorder = Recorder() + let controller = try makeController(recorder: recorder) + controller.bridge.deliverToolInput(arguments: ["startDate": .string("today")]) + controller.bridge.deliverToolResult(MCPToolResult(text: "t", structuredContent: .object(["totalCount": .number(3)]))) + await controller.load() + let finished = await waitForReport(recorder) + let r = reports(recorder) + await controller.close() + + #expect(finished, "widget never finished its probes; got \(r)") + // The bridge works through the sandboxed iframe + relay. + #expect(r["protocolVersion"] == "2026-01-26") + #expect(r["csp-connect"] == "[]") + #expect(r["tool-input"] == #"{"startDate":"today"}"#) + #expect(r["tool-result"]?.contains("\"totalCount\":3") == true) + // Network is closed at every layer. + #expect(r["fetch"] == "blocked") + #expect(r["ws"] == "blocked") + #expect(r["img"] == "blocked") + // The iframe is an opaque origin: no parent DOM, no storage. + #expect(r["parent-dom"] == "blocked") + #expect(r["storage"] == "blocked") + // A direct call to the native handler from the widget frame is dropped. + #expect(r["direct-handler"] == nil) + // Denied by default: link and message never reached the host actions. + #expect(recorder.openedLinks.isEmpty) + #expect(recorder.audit.contains { $0.method == "ui/open-link" && $0.outcome == .denied }) + #expect(recorder.audit.contains { $0.method == "ui/message" && $0.outcome == .denied }) + } + + @Test func closeTearsDownAndStopsAcceptingMessages() async throws { + let recorder = Recorder() + let controller = try makeController(recorder: recorder) + await controller.load() + _ = await waitForReport(recorder) + await controller.close() + #expect(controller.bridge.state == .tornDown) + } + + @Test func fullscreenIsGrantedNotifiedAndCanBeExited() async throws { + let recorder = Recorder() + let controller = try makeController(recorder: recorder, html: contextWidget, modes: ["inline", "fullscreen"]) + await controller.load() + #expect(await waitForReport(recorder, key: "mode-change")) + var r = reports(recorder) + #expect(r["modes"] == #"["inline","fullscreen"]"#) + #expect(r["mode-reply"] == "fullscreen") + #expect(r["mode-change"] == "fullscreen") + #expect(controller.displayMode == "fullscreen") + // The host leaves fullscreen; the widget is told. + recorder.calls.removeAll() + controller.exitFullscreen() + #expect(await waitForReport(recorder, key: "mode-change")) + r = reports(recorder) + #expect(r["mode-change"] == "inline") + #expect(controller.displayMode == "inline") + await controller.close() + } + + @Test func widgetOnlySeesFullscreenWhenTheHostSupportsIt() async throws { + let recorder = Recorder() + let controller = try makeController(recorder: recorder, html: contextWidget) // inline only + await controller.load() + #expect(await waitForReport(recorder, key: "mode-reply")) + let r = reports(recorder) + #expect(r["modes"] == #"["inline"]"#) + #expect(r["mode-reply"] == "inline") // the request is politely refused + #expect(controller.displayMode == "inline") + await controller.close() + } + + @Test func containerSizeAndThemeReachTheWidget() async throws { + let recorder = Recorder() + let controller = try makeController(recorder: recorder, html: contextWidget) + let window = NSWindow(contentRect: NSRect(x: 0, y: 0, width: 500, height: 400), styleMask: [.titled], backing: .buffered, defer: false) + window.contentView = controller.webView + window.orderFrontRegardless() + await controller.load() + #expect(await waitForReport(recorder, key: "theme0")) + #expect(["light", "dark"].contains(reports(recorder)["theme0"] ?? "")) + // Resize the container after the widget is up: it must hear about the new size. + recorder.calls.removeAll() + window.setContentSize(NSSize(width: 700, height: 520)) + #expect(await waitForReport(recorder, key: "dims")) + #expect(reports(recorder)["dims"] == "700x520") + // Switch appearance: the widget must hear about the theme. + recorder.calls.removeAll() + window.appearance = NSAppearance(named: .darkAqua) + controller.webView.appearance = NSAppearance(named: .darkAqua) + #expect(await waitForReport(recorder, key: "theme-change")) + #expect(reports(recorder)["theme-change"] == "dark") + await controller.close() + window.orderOut(nil) + } +} diff --git a/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsBridgeTests.swift b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsBridgeTests.swift new file mode 100644 index 0000000..971b50f --- /dev/null +++ b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsBridgeTests.swift @@ -0,0 +1,438 @@ +import Foundation +import LocalLMLabSDKCore +import Testing + +import LocalLMLabSDKMCPAppsHost + +// Drives the bridge exactly as a web-view layer would: raw JSON-RPC bytes in, serialized +// messages out. Public API only. + +@MainActor +private final class Recorder { + var sent: [[String: Any]] = [] + var audit: [MCPAppsAuditEvent] = [] + var toolCalls: [(name: String, arguments: [String: MCPValue])] = [] + var resourceReads: [String] = [] + var sizes: [(Double?, Double?)] = [] +} + +@MainActor +private struct FakeBackend: MCPAppsBackend { + let recorder: Recorder + var toolResult: Result = .success( + MCPToolResult(text: "ok", structuredContent: .object(["tasks": .array([])]), meta: .object(["k": .string("v")]))) + + func callTool(name: String, arguments: [String: MCPValue]) async -> Result { + recorder.toolCalls.append((name, arguments)) + return toolResult + } + + func readResource(uri: String) async -> Result { + recorder.resourceReads.append(uri) + return .success(MCPResourceContent(uri: uri, mimeType: "text/html;profile=mcp-app", text: "")) + } +} + +@MainActor +private struct OpenPolicy: MCPAppsBridgePolicy { + func authorizeOpenLink(_ url: URL) async -> MCPAppsDecision { .allow } + func authorizeMessage(text: String) async -> MCPAppsDecision { .allow } + func authorizeModelContextUpdate() async -> MCPAppsDecision { .allow } +} + +private func tool(_ name: String, visibility: [String]? = nil) -> MCPToolDescriptor { + let meta: MCPValue? = visibility.map { .object(["ui": .object(["visibility": .array($0.map { .string($0) })])]) } + return MCPToolDescriptor( + serverID: MCPServerID(rawValue: "https://example.invalid/mcp"), name: name, description: "", + rawSchema: Data("{}".utf8), estimatedTokens: 0, meta: meta) +} + +@MainActor +private struct Harness { + let recorder = Recorder() + let bridge: MCPAppsBridge + + init( + tools: [MCPToolDescriptor] = [tool("find-tasks"), tool("complete-tasks"), tool("model-only", visibility: ["model"])], + policy: (any MCPAppsBridgePolicy)? = nil, + backend: ((Recorder) -> FakeBackend)? = nil, + configuration: MCPAppsBridgeConfiguration = MCPAppsBridgeConfiguration(hostName: "TestHost", hostVersion: "1.0"), + actions: MCPAppsHostActions = MCPAppsHostActions() + ) { + let recorder = self.recorder + bridge = MCPAppsBridge( + configuration: configuration, tools: tools, + backend: backend?(recorder) ?? FakeBackend(recorder: recorder), + policy: policy ?? MCPAppsDefaultPolicy(), + send: { data in + if let object = (try? JSONSerialization.jsonObject(with: data)) as? [String: Any] { recorder.sent.append(object) } + }, + audit: { recorder.audit.append($0) }, + onSizeChanged: { recorder.sizes.append(($0, $1)) }, + actions: actions) + } + + func post(_ object: [String: Any]) async { + await bridge.receive(try! JSONSerialization.data(withJSONObject: object)) + } + + func request(_ id: Any, _ method: String, _ params: [String: Any] = [:]) async { + await post(["jsonrpc": "2.0", "id": id, "method": method, "params": params]) + } + + func notify(_ method: String, _ params: [String: Any] = [:]) async { + await post(["jsonrpc": "2.0", "method": method, "params": params]) + } + + /// The standard opening handshake. + func initialize() async { + await request(1, "ui/initialize", ["protocolVersion": "2026-01-26", "clientInfo": ["name": "w", "version": "1"], "appCapabilities": [:]]) + await notify("ui/notifications/initialized") + } + + func reply(to id: Int) -> [String: Any]? { + recorder.sent.first { ($0["id"] as? Int) == id && ($0["result"] != nil || $0["error"] != nil) } + } + + func errorCode(for id: Int) -> Int? { (reply(to: id)?["error"] as? [String: Any])?["code"] as? Int } + func result(for id: Int) -> [String: Any]? { reply(to: id)?["result"] as? [String: Any] } +} + +@MainActor +@Suite struct MCPAppsBridgeTests { + // MARK: lifecycle + + @Test func initializeAnswersWithHostInfoAndClosedCapabilities() async throws { + let h = Harness() + await h.request(1, "ui/initialize", ["protocolVersion": "2026-01-26"]) + let result = try #require(h.result(for: 1)) + #expect(result["protocolVersion"] as? String == "2026-01-26") + #expect((result["hostInfo"] as? [String: Any])?["name"] as? String == "TestHost") + let caps = try #require(result["hostCapabilities"] as? [String: Any]) + #expect(caps["serverTools"] != nil) + #expect(caps["openLinks"] == nil) // we deny links by default: don't advertise it + let csp = try #require((caps["sandbox"] as? [String: Any])?["csp"] as? [String: Any]) + #expect((csp["connectDomains"] as? [String])?.isEmpty == true) + let context = try #require(result["hostContext"] as? [String: Any]) + #expect(context["displayMode"] as? String == "inline") + #expect(h.bridge.state == .awaitingInitialized) + await h.notify("ui/notifications/initialized") + #expect(h.bridge.state == .ready) + } + + @Test func initializeRequiresProtocolVersion() async { + let h = Harness() + await h.request(1, "ui/initialize", [:]) + #expect(h.errorCode(for: 1) == -32602) + #expect(h.bridge.state == .awaitingInitialize) + } + + @Test func secondInitializeIsRefused() async { + let h = Harness() + await h.initialize() + await h.request(2, "ui/initialize", ["protocolVersion": "2026-01-26"]) + #expect(h.errorCode(for: 2) == -32600) + } + + @Test func requestsBeforeInitializedAreRefusedAndNeverReachTheBackend() async { + let h = Harness() + await h.request(1, "tools/call", ["name": "find-tasks"]) + #expect(h.errorCode(for: 1) == -32600) + await h.request(2, "ui/initialize", ["protocolVersion": "2026-01-26"]) + await h.request(3, "tools/call", ["name": "find-tasks"]) // initialized notification not yet sent + #expect(h.errorCode(for: 3) == -32600) + #expect(h.recorder.toolCalls.isEmpty) + } + + @Test func hostNotificationsAreQueuedUntilInitializedThenFlushedInOrder() async { + let h = Harness() + h.bridge.deliverToolInput(arguments: ["startDate": .string("today")]) + h.bridge.deliverToolResult(MCPToolResult(text: "t", structuredContent: .object(["n": .number(1)]))) + #expect(h.recorder.sent.isEmpty) + await h.request(1, "ui/initialize", ["protocolVersion": "2026-01-26"]) + let afterInit = h.recorder.sent.count // just the initialize reply + #expect(afterInit == 1) + await h.notify("ui/notifications/initialized") + let methods = h.recorder.sent.dropFirst(afterInit).compactMap { $0["method"] as? String } + #expect(methods == ["ui/notifications/tool-input", "ui/notifications/tool-result"]) + } + + @Test func toolResultNotificationCarriesStructuredContentAndMeta() async throws { + let h = Harness() + await h.initialize() + h.bridge.deliverToolResult(MCPToolResult(text: "hi", structuredContent: .object(["totalCount": .number(2)]), isError: false, meta: .object(["a": .string("b")]))) + let params = try #require(h.recorder.sent.last?["params"] as? [String: Any]) + #expect((params["structuredContent"] as? [String: Any])?["totalCount"] as? Int == 2) + #expect((params["_meta"] as? [String: Any])?["a"] as? String == "b") + #expect(((params["content"] as? [[String: Any]])?.first)?["text"] as? String == "hi") + } + + // MARK: tools/call + + @Test func toolCallReachesTheBackendAndReturnsTheResult() async throws { + let h = Harness() + await h.initialize() + await h.request(2, "tools/call", ["name": "complete-tasks", "arguments": ["ids": ["a", "b"], "n": 3]]) + #expect(h.recorder.toolCalls.count == 1) + #expect(h.recorder.toolCalls[0].name == "complete-tasks") + #expect(h.recorder.toolCalls[0].arguments["ids"] == .array([.string("a"), .string("b")])) + let result = try #require(h.result(for: 2)) + #expect((result["structuredContent"] as? [String: Any])?["tasks"] != nil) + #expect((result["_meta"] as? [String: Any])?["k"] as? String == "v") + } + + @Test func unknownToolIsRefused() async { + let h = Harness() + await h.initialize() + await h.request(2, "tools/call", ["name": "delete-everything"]) + #expect(h.errorCode(for: 2) == -32000) + #expect(h.recorder.toolCalls.isEmpty) + } + + @Test func modelOnlyToolIsNotCallableByTheView() async { + let h = Harness() + await h.initialize() + await h.request(2, "tools/call", ["name": "model-only"]) + #expect(h.errorCode(for: 2) == -32000) + #expect(h.recorder.toolCalls.isEmpty) + } + + @Test func policyAllowlistBlocksUnapprovedTools() async { + let h = Harness(policy: MCPAppsDefaultPolicy(allowedTools: ["find-tasks"])) + await h.initialize() + await h.request(2, "tools/call", ["name": "complete-tasks"]) + await h.request(3, "tools/call", ["name": "find-tasks"]) + #expect(h.errorCode(for: 2) == -32000) + #expect(h.reply(to: 3)?["result"] != nil) + #expect(h.recorder.toolCalls.map(\.name) == ["find-tasks"]) + } + + @Test func toolCallValidatesParams() async { + let h = Harness() + await h.initialize() + await h.request(2, "tools/call", [:]) + await h.request(3, "tools/call", ["name": "find-tasks", "arguments": "not an object"]) + #expect(h.errorCode(for: 2) == -32602) + #expect(h.errorCode(for: 3) == -32602) + #expect(h.recorder.toolCalls.isEmpty) + } + + @Test func backendFailureBecomesAnErrorWithoutLeakingDetail() async throws { + let h = Harness(backend: { FakeBackend(recorder: $0, toolResult: .failure(.serverError("secret token abc123"))) }) + await h.initialize() + await h.request(2, "tools/call", ["name": "find-tasks"]) + let error = try #require(h.reply(to: 2)?["error"] as? [String: Any]) + #expect(error["code"] as? Int == -32000) + #expect(!(error["message"] as? String ?? "").contains("abc123")) + } + + // MARK: things a view must not do by default + + @Test func messageContextAndLinksAreDeniedByDefault() async { + let h = Harness() + await h.initialize() + await h.request(2, "ui/message", ["role": "user", "content": ["type": "text", "text": "hi model"]]) + await h.request(3, "ui/update-model-context", ["content": [["type": "text", "text": "x"]]]) + await h.request(4, "ui/open-link", ["url": "https://example.com"]) + #expect(h.errorCode(for: 2) == -32000) + #expect(h.errorCode(for: 3) == -32000) + #expect(h.errorCode(for: 4) == -32000) + } + + @Test func aPolicyCanOptIn() async { + let h = Harness(policy: OpenPolicy()) + await h.initialize() + await h.request(2, "ui/message", ["role": "user", "content": ["type": "text", "text": "hi"]]) + await h.request(3, "ui/update-model-context", ["content": []]) + await h.request(4, "ui/open-link", ["url": "https://example.com"]) + #expect(h.reply(to: 2)?["result"] != nil) + #expect(h.reply(to: 3)?["result"] != nil) + #expect(h.reply(to: 4)?["result"] != nil) + } + + @Test func openLinkRejectsNonWebSchemesEvenWhenThePolicyAllows() async { + let h = Harness(policy: OpenPolicy()) + await h.initialize() + await h.request(2, "ui/open-link", ["url": "file:///etc/passwd"]) + await h.request(3, "ui/open-link", ["url": "javascript:alert(1)"]) + await h.request(4, "ui/open-link", ["url": "x-apple.systempreferences:"]) + #expect(h.errorCode(for: 2) == -32000) + #expect(h.errorCode(for: 3) == -32000) + #expect(h.errorCode(for: 4) == -32000) + } + + @Test func nonUIResourceReadsAreDeniedByDefault() async { + let h = Harness() + await h.initialize() + await h.request(2, "resources/read", ["uri": "file:///etc/passwd"]) + await h.request(3, "resources/read", ["uri": "ui://todoist/task-list@abc"]) + #expect(h.errorCode(for: 2) == -32000) + #expect(h.result(for: 3)?["contents"] != nil) + #expect(h.recorder.resourceReads == ["ui://todoist/task-list@abc"]) + } + + @Test func displayModeRequestsKeepTheCurrentMode() async { + let h = Harness() + await h.initialize() + await h.request(2, "ui/request-display-mode", ["mode": "fullscreen"]) + #expect(h.result(for: 2)?["mode"] as? String == "inline") + } + + // MARK: display modes + host context + + @Test func fullscreenIsOnlyAdvertisedWhenTheHostProvidesIt() async throws { + let plain = Harness() + await plain.request(1, "ui/initialize", ["protocolVersion": "2026-01-26"]) + let modes = (plain.result(for: 1)?["hostContext"] as? [String: Any])?["availableDisplayModes"] as? [String] + #expect(modes == ["inline"]) + let rich = Harness(configuration: MCPAppsBridgeConfiguration(hostName: "H", hostVersion: "1", availableDisplayModes: ["inline", "fullscreen"])) + await rich.request(1, "ui/initialize", ["protocolVersion": "2026-01-26"]) + #expect(((rich.result(for: 1)?["hostContext"] as? [String: Any])?["availableDisplayModes"] as? [String]) == ["inline", "fullscreen"]) + } + + @Test func aGrantedDisplayModeRequestSwitchesModeAndNotifiesTheView() async { + var requested: [String] = [] + let h = Harness(configuration: MCPAppsBridgeConfiguration(hostName: "H", hostVersion: "1", availableDisplayModes: ["inline", "fullscreen"]), + actions: MCPAppsHostActions(requestDisplayMode: { requested.append($0); return true })) + await h.initialize() + await h.request(2, "ui/request-display-mode", ["mode": "fullscreen"]) + #expect(h.result(for: 2)?["mode"] as? String == "fullscreen") + #expect(requested == ["fullscreen"]) + #expect(h.bridge.displayMode == "fullscreen") + let change = h.recorder.sent.last { $0["method"] as? String == "ui/notifications/host-context-changed" } + #expect((change?["params"] as? [String: Any])?["displayMode"] as? String == "fullscreen") + } + + @Test func aDeclinedOrUnadvertisedDisplayModeKeepsTheCurrentMode() async { + var asked = 0 + let h = Harness(configuration: MCPAppsBridgeConfiguration(hostName: "H", hostVersion: "1", availableDisplayModes: ["inline", "fullscreen"]), + actions: MCPAppsHostActions(requestDisplayMode: { _ in asked += 1; return false })) + await h.initialize() + await h.request(2, "ui/request-display-mode", ["mode": "fullscreen"]) // host declines + await h.request(3, "ui/request-display-mode", ["mode": "pip"]) // never advertised + await h.request(4, "ui/request-display-mode", ["mode": 7]) // malformed + for id in 2...4 { #expect(h.result(for: id)?["mode"] as? String == "inline") } + #expect(asked == 1) // only the advertised mode reached the host + #expect(!h.recorder.sent.contains { $0["method"] as? String == "ui/notifications/host-context-changed" }) + } + + @Test func theHostCanChangeTheModeItself() async { + let h = Harness(configuration: MCPAppsBridgeConfiguration(hostName: "H", hostVersion: "1", availableDisplayModes: ["inline", "fullscreen"])) + await h.initialize() + h.bridge.setDisplayMode("fullscreen") + h.bridge.setDisplayMode("fullscreen") // no-op the second time + h.bridge.setDisplayMode("pip") // not advertised: ignored + let changes = h.recorder.sent.filter { $0["method"] as? String == "ui/notifications/host-context-changed" } + #expect(changes.count == 1) + #expect(h.bridge.displayMode == "fullscreen") + } + + @Test func hostContextChangesBeforeReadyAreMergedIntoOneNotification() async throws { + let h = Harness() + h.bridge.deliverHostContextChange(["theme": .string("light"), "containerDimensions": .object(["width": .number(100)])]) + h.bridge.deliverHostContextChange(["containerDimensions": .object(["width": .number(640)])]) // later wins + h.bridge.deliverHostContextChange(["theme": .string("dark")]) + #expect(h.recorder.sent.isEmpty) + await h.initialize() + let changes = h.recorder.sent.filter { $0["method"] as? String == "ui/notifications/host-context-changed" } + #expect(changes.count == 1) + let params = try #require(changes.first?["params"] as? [String: Any]) + #expect(params["theme"] as? String == "dark") + #expect((params["containerDimensions"] as? [String: Any])?["width"] as? Int == 640) + } + + @Test func hostContextChangesAfterReadyGoStraightOut() async { + let h = Harness() + await h.initialize() + h.bridge.deliverHostContextChange(["theme": .string("dark")]) + #expect(h.recorder.sent.contains { $0["method"] as? String == "ui/notifications/host-context-changed" }) + } + + // MARK: hostile / malformed input + + @Test func unknownMethodsGetMethodNotFound() async { + let h = Harness() + await h.initialize() + await h.request(2, "sampling/createMessage", [:]) + #expect(h.errorCode(for: 2) == -32601) + } + + @Test func malformedAndOversizedInputIsDroppedNotFatal() async { + let config = MCPAppsBridgeConfiguration(hostName: "H", hostVersion: "1", maxMessageBytes: 200) + let h = Harness(configuration: config) + await h.bridge.receive(Data("not json".utf8)) + await h.bridge.receive(Data(#"{"method":"ping","id":1}"#.utf8)) // missing jsonrpc + await h.bridge.receive(Data(String(repeating: " ", count: 500).utf8)) + await h.post(["jsonrpc": "2.0", "id": ["nested": true], "method": "ping"]) // invalid id type + #expect(h.recorder.sent.isEmpty) + #expect(h.recorder.audit.filter { $0.outcome == .dropped }.count == 4) + await h.request(1, "ping") // still works afterwards + #expect(h.reply(to: 1) != nil) + } + + @Test func booleanIDsAreNotAcceptedAsNumbers() async { + let h = Harness() + await h.post(["jsonrpc": "2.0", "id": true, "method": "ping"]) + #expect(h.recorder.sent.isEmpty) + } + + @Test func sizeChangesAreReported() async { + let h = Harness() + await h.initialize() + await h.notify("ui/notifications/size-changed", ["width": 400, "height": 320]) + #expect(h.recorder.sizes.count == 1) + #expect(h.recorder.sizes[0].0 == 400) + #expect(h.recorder.sizes[0].1 == 320) + } + + // MARK: teardown + + @Test func teardownWaitsForTheViewsAnswerThenClosesTheBridge() async { + let h = Harness() + await h.initialize() + let task = Task { await h.bridge.teardown(reason: "closing") } + while !h.recorder.sent.contains(where: { $0["method"] as? String == "ui/resource-teardown" }) { await Task.yield() } + let request = h.recorder.sent.first { $0["method"] as? String == "ui/resource-teardown" }! + await h.post(["jsonrpc": "2.0", "id": request["id"]!, "result": [:]]) + await task.value + #expect(h.bridge.state == .tornDown) + } + + @Test func teardownTimesOutIfTheViewIsSilent() async { + let config = MCPAppsBridgeConfiguration(hostName: "H", hostVersion: "1", teardownTimeout: .milliseconds(50)) + let h = Harness(configuration: config) + await h.initialize() + await h.bridge.teardown() + #expect(h.bridge.state == .tornDown) + } + + @Test func nothingIsAcceptedOrSentAfterTeardown() async { + let config = MCPAppsBridgeConfiguration(hostName: "H", hostVersion: "1", teardownTimeout: .milliseconds(20)) + let h = Harness(configuration: config) + await h.initialize() + await h.bridge.teardown() + let sentBefore = h.recorder.sent.count + await h.request(9, "tools/call", ["name": "find-tasks"]) + h.bridge.deliverToolInput(arguments: [:]) + #expect(h.recorder.sent.count == sentBefore) + #expect(h.recorder.toolCalls.isEmpty) + } + + // MARK: audit + + @Test func auditNeverRecordsArgumentValuesOrOutput() async { + let h = Harness() + await h.initialize() + await h.request(2, "tools/call", ["name": "complete-tasks", "arguments": ["secret": "hunter2-do-not-log"]]) + let dump = h.recorder.audit.map { "\($0.method)|\($0.outcome)|\($0.detail ?? "")" }.joined(separator: "\n") + #expect(!dump.contains("hunter2")) + #expect(dump.contains("complete-tasks")) // the tool NAME is recorded + } + + @Test func deniedCallsAreAuditedAsDenied() async { + let h = Harness() + await h.initialize() + await h.request(2, "ui/message", ["role": "user", "content": ["type": "text", "text": "x"]]) + #expect(h.recorder.audit.contains { $0.method == "ui/message" && $0.outcome == .denied }) + } +} diff --git a/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSandboxTests.swift b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSandboxTests.swift new file mode 100644 index 0000000..362bc7b --- /dev/null +++ b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSandboxTests.swift @@ -0,0 +1,156 @@ +import Foundation +import Testing + +import LocalLMLabSDKMCPAppsHost + +@Suite struct MCPAppsSandboxTests { + // MARK: origin validation (server-supplied strings go into a CSP header) + + @Test func acceptsPlainHTTPSAndWSSOrigins() { + for ok in ["https://api.example.com", "https://a.b.example.co.uk:8443", "wss://ws.example.com", "https://*.example.com", "HTTPS://API.Example.COM"] { + #expect(MCPAppsSandbox.isSafeOrigin(ok), "\(ok)") + } + } + + @Test func rejectsAnythingThatCouldInjectOrWeaken() { + let bad = [ + "http://example.com", // no cleartext + "https://example.com/path", // not an origin + "https://example.com; script-src *", // directive injection + "https://example.com script-src *", + "https://example.com,https://evil.com", + "'unsafe-eval'", "*", "data:", "blob:", "self", "https://*", "https://*.com.", + "https://localhost", "https://com", "https://-bad.example.com", "https://ex ample.com", + "https://example.com:0", "https://example.com:99999", "https://example.com:", + "https://exämple.com", "", "https://", "https://.example.com", "https://example..com", + "https://example.com\r\nX-Evil: 1", "https://exa\"mple.com", + ] + for origin in bad { #expect(!MCPAppsSandbox.isSafeOrigin(origin), "should reject \(origin.debugDescription)") } + } + + // MARK: intersection + + @Test func closedPolicyGrantsNothingWhateverTheServerAsks() { + let declared = MCPAppCSP( + connectDomains: ["https://api.example.com"], resourceDomains: ["https://cdn.example.com"], + frameDomains: ["https://embed.example.com"], baseUriDomains: ["https://base.example.com"]) + let effective = MCPAppsSandbox.effectiveCSP(declared: declared, policy: .closed) + #expect(effective.isEmpty) + } + + @Test func exactAllowlistEntriesGrantOnlyMatchingDeclaredOrigins() { + let policy = MCPAppsSandboxPolicy(allowedConnectOrigins: ["https://api.example.com"]) + let declared = MCPAppCSP(connectDomains: ["https://api.example.com", "https://evil.example.net", "https://API.example.com:8443"]) + #expect(MCPAppsSandbox.effectiveCSP(declared: declared, policy: policy).connectDomains == ["https://api.example.com"]) + } + + @Test func eachDirectiveUsesItsOwnAllowlist() { + let policy = MCPAppsSandboxPolicy(allowedConnectOrigins: ["https://api.example.com"]) + let declared = MCPAppCSP(connectDomains: ["https://api.example.com"], resourceDomains: ["https://api.example.com"]) + let effective = MCPAppsSandbox.effectiveCSP(declared: declared, policy: policy) + #expect(effective.connectDomains == ["https://api.example.com"]) + #expect(effective.resourceDomains.isEmpty) // connect allowance doesn't leak into resources + } + + @Test func wildcardAllowlistCoversSubdomainsButNotTheApexOrLookalikes() { + let policy = MCPAppsSandboxPolicy(allowedResourceOrigins: ["https://*.example.com"]) + let declared = MCPAppCSP(resourceDomains: [ + "https://cdn.example.com", "https://a.b.example.com", "https://example.com", + "https://evilexample.com", "https://cdn.example.com.evil.net", "wss://cdn.example.com", "https://cdn.example.com:8443", + ]) + #expect(MCPAppsSandbox.effectiveCSP(declared: declared, policy: policy).resourceDomains + == ["https://cdn.example.com", "https://a.b.example.com"]) + } + + @Test func aDeclaredWildcardNeedsAnIdenticalWildcardInPolicy() { + let declared = MCPAppCSP(connectDomains: ["https://*.example.com"]) + let exactOnly = MCPAppsSandboxPolicy(allowedConnectOrigins: ["https://api.example.com"]) + #expect(MCPAppsSandbox.effectiveCSP(declared: declared, policy: exactOnly).connectDomains.isEmpty) + let wildcard = MCPAppsSandboxPolicy(allowedConnectOrigins: ["https://*.example.com"]) + #expect(MCPAppsSandbox.effectiveCSP(declared: declared, policy: wildcard).connectDomains == ["https://*.example.com"]) + } + + @Test func baseURIIsNeverGranted() { + let policy = MCPAppsSandboxPolicy(allowedConnectOrigins: ["https://a.example.com"], allowedResourceOrigins: ["https://a.example.com"]) + let declared = MCPAppCSP(baseUriDomains: ["https://a.example.com"]) + #expect(MCPAppsSandbox.effectiveCSP(declared: declared, policy: policy).baseUriDomains.isEmpty) + } + + @Test func intersectionDeduplicatesAndNormalizes() { + let policy = MCPAppsSandboxPolicy(allowedConnectOrigins: ["https://api.example.com"]) + let declared = MCPAppCSP(connectDomains: ["https://API.example.com", "https://api.example.com"]) + #expect(MCPAppsSandbox.effectiveCSP(declared: declared, policy: policy).connectDomains == ["https://api.example.com"]) + } + + // MARK: header + + @Test func emptyCSPHeaderIsFullyClosed() { + let header = MCPAppsSandbox.headerValue(for: MCPAppCSP()) + #expect(header.contains("default-src 'none'")) + #expect(header.contains("connect-src 'none'")) + #expect(header.contains("frame-src 'none'")) + #expect(header.contains("base-uri 'none'")) + #expect(header.contains("form-action 'none'")) + #expect(header.contains("object-src 'none'")) + #expect(!header.contains("http")) + #expect(!header.contains("'unsafe-eval'")) + } + + @Test func headerListsGrantedOriginsInTheRightDirectives() { + let csp = MCPAppCSP( + connectDomains: ["https://api.example.com"], resourceDomains: ["https://cdn.example.com"], + frameDomains: ["https://embed.example.com"]) + let header = MCPAppsSandbox.headerValue(for: csp) + #expect(header.contains("connect-src https://api.example.com;")) + #expect(header.contains("script-src 'self' 'unsafe-inline' https://cdn.example.com;")) + #expect(header.contains("img-src 'self' data: blob: https://cdn.example.com;")) + #expect(header.contains("frame-src https://embed.example.com;")) + #expect(!header.contains("connect-src https://api.example.com https://cdn")) + } + + @Test func headerRefusesToEmitAnUnsafeOriginEvenIfHandedOne() { + let hostile = MCPAppCSP( + connectDomains: ["https://ok.example.com", "https://x.com; script-src *"], + resourceDomains: ["https://y.com 'unsafe-eval'"], frameDomains: ["*"]) + let header = MCPAppsSandbox.headerValue(for: hostile) + #expect(header.contains("https://ok.example.com")) + #expect(!header.contains("script-src *")) + #expect(!header.contains("unsafe-eval")) + #expect(!header.contains("x.com")) + #expect(header.contains("frame-src 'none'")) + } + + // MARK: WebKit block list + + private func rules(_ csp: MCPAppCSP) -> [[String: [String: String]]] { + let data = Data(MCPAppsSandbox.contentRuleListJSON(for: csp).utf8) + return (try! JSONSerialization.jsonObject(with: data)) as! [[String: [String: String]]] + } + + @Test func blockListBlocksAllNetworkSchemesFirst() { + let r = rules(MCPAppCSP()) + #expect(r.count == 3) + #expect(r.allSatisfy { $0["action"]?["type"] == "block" }) + let filters = r.compactMap { $0["trigger"]?["url-filter"] } + #expect(filters.contains("^https?://")) + #expect(filters.contains("^wss?://")) + #expect(filters.contains("^ftp://")) + } + + @Test func blockListExemptsOnlyEffectiveOriginsAfterTheBlocks() { + let r = rules(MCPAppCSP(connectDomains: ["https://api.example.com"], resourceDomains: ["https://*.cdn.example.com", "https://x.example.org:8443"])) + #expect(r.prefix(3).allSatisfy { $0["action"]?["type"] == "block" }) + let allows = r.dropFirst(3) + #expect(allows.allSatisfy { $0["action"]?["type"] == "ignore-previous-rules" }) + let filters = Set(allows.compactMap { $0["trigger"]?["url-filter"] }) + #expect(filters.contains("^https://api\\.example\\.com/")) + #expect(filters.contains("^https://[a-z0-9.-]+\\.cdn\\.example\\.com/")) + #expect(filters.contains("^https://x\\.example\\.org:8443/")) + #expect(filters.count == 3) + } + + @Test func blockListNeverExemptsAnUnsafeOrigin() { + let r = rules(MCPAppCSP(connectDomains: ["https://*", "http://example.com", "https://a.com/x"])) + #expect(r.count == 3) // only the three blocks + } +} diff --git a/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSessionBackendTests.swift b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSessionBackendTests.swift new file mode 100644 index 0000000..30ac4a7 --- /dev/null +++ b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPAppsSessionBackendTests.swift @@ -0,0 +1,45 @@ +import Foundation +import LocalLMLabSDKCore +import Testing + +import LocalLMLabSDKMCPAppsHost + +// The session-backed adapter for model-present hosts. Public API only; the session behavior +// itself is tested in the SDK. + +@MainActor +@Suite struct MCPAppsSessionBackendTests { + private func session() throws -> (LocalLMLab, LocalLMLabSession) { + let lab = LocalLMLab(configuration: .init(providers: [SystemModelProvider()])) + lab.models.route(.light, to: .system) + return (lab, try lab.makeSession(route: .light)) + } + + @Test func widgetContextReachesTheSessionAsText() throws { + let (_, s) = try session() + let actions = MCPAppsHostActions.session(s, instance: "w1") + actions.updateModelContext( + [.object(["type": .string("text"), "text": .string("Selected 2 tasks")])], + .object(["ids": .array([.string("a"), .string("b")])])) + let entry = try #require(s.hostTranscript.entries.first) + #expect(entry.appInstance == "w1") + guard case .appContext(let instance, let text) = entry.content else { Issue.record("expected an app-context entry"); return } + #expect(instance == "w1") + #expect(text == "Selected 2 tasks\n{\"ids\":[\"a\",\"b\"]}") + } + + @Test func handlersCanDeclareMCPAppsSupport() { + let handlers = MCPClientHandlers(extensions: ["other": .bool(true)]).advertisingMCPApps() + #expect(handlers.extensions["other"] == .bool(true)) + #expect(handlers.extensions[MCPClientHandlers.mcpAppsExtensionKey] + == .object(["mimeTypes": .array([.string(MCPAppResource.mimeType)])])) + } + + @Test func aCallToAServerTheSessionDoesNotKnowIsNotFound() async throws { + let (lab, s) = try session() + let backend = MCPAppsSessionBackend(session: s, manager: lab.mcp, server: MCPServerID(rawValue: "https://nowhere.invalid/mcp"), instance: "w1") + let result = await backend.callTool(name: "find", arguments: [:]) + guard case .failure(.toolNotFound) = result else { Issue.record("expected .toolNotFound, got \(result)"); return } + #expect(s.hostTranscript.entries.isEmpty) + } +} diff --git a/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPToolInputFormTests.swift b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPToolInputFormTests.swift new file mode 100644 index 0000000..3dfebbf --- /dev/null +++ b/MCPAppsHost/Tests/LocalLMLabSDKMCPAppsHostTests/MCPToolInputFormTests.swift @@ -0,0 +1,123 @@ +import Foundation +import LocalLMLabSDKCore +import Testing + +import LocalLMLabSDKMCPAppsHost + +@Suite struct MCPToolInputFormTests { + private func form(_ schema: String) -> MCPToolInputForm { MCPToolInputForm(schema: Data(schema.utf8)) } + + @Test func toolsWithoutInputsHaveAnEmptyForm() { + #expect(form(#"{"type":"object","properties":{}}"#).isEmpty) + #expect(form("{}").isEmpty) + #expect(form("not json").isEmpty) + #expect(form(#"{"type":"object"}"#).isEmpty) + } + + @Test func mapsTheCommonTypes() { + let f = form(#""" + {"type":"object","required":["startDate"],"properties":{ + "startDate":{"type":"string","description":"When","pattern":"^x$"}, + "limit":{"type":"integer","default":10,"minimum":1}, + "ratio":{"type":"number"}, + "dryRun":{"type":"boolean","default":false}, + "mode":{"type":"string","enum":["a","b"]}, + "labels":{"type":"array","items":{"type":"string"}}, + "ids":{"type":"array","items":{"type":"integer"}}, + "filter":{"type":"object","properties":{}}, + "either":{"anyOf":[{"type":"string"},{"type":"integer"}]} + }} + """#) + let byName = Dictionary(uniqueKeysWithValues: f.fields.map { ($0.name, $0) }) + #expect(byName["startDate"]?.kind == .string) + #expect(byName["startDate"]?.isRequired == true) + #expect(byName["startDate"]?.summary == "When") + #expect(byName["limit"]?.kind == .integer) + #expect(byName["limit"]?.defaultText == "10") + #expect(byName["ratio"]?.kind == .number) + #expect(byName["dryRun"]?.kind == .boolean) + #expect(byName["dryRun"]?.defaultText == "false") + #expect(byName["mode"]?.kind == .choice(["a", "b"])) + #expect(byName["labels"]?.kind == .list(element: .string)) + #expect(byName["ids"]?.kind == .list(element: .integer)) + #expect(byName["filter"]?.kind == .json) + #expect(byName["either"]?.kind == .json) + #expect(f.hasRequiredFields) + } + + @Test func requiredFieldsComeFirstThenAlphabetical() { + let f = form(#"{"required":["z"],"properties":{"b":{"type":"string"},"a":{"type":"string"},"z":{"type":"string"}}}"#) + #expect(f.fields.map(\.name) == ["z", "a", "b"]) + } + + @Test func nullableTypeListsResolveToTheRealType() { + let f = form(#"{"properties":{"x":{"type":["string","null"]},"y":{"type":["string","integer"]}}}"#) + #expect(f.fields.first { $0.name == "x" }?.kind == .string) + #expect(f.fields.first { $0.name == "y" }?.kind == .json) // ambiguous: raw JSON + } + + @Test func nonStringEnumsFallBackToJSON() { + #expect(form(#"{"properties":{"n":{"enum":[1,2,3]}}}"#).fields[0].kind == .json) + } + + @Test func buildsTypedArgumentsAndOmitsBlankOptionals() throws { + let f = form(#""" + {"required":["startDate"],"properties":{"startDate":{"type":"string"},"limit":{"type":"integer"},"ratio":{"type":"number"}, + "dryRun":{"type":"boolean"},"mode":{"type":"string","enum":["a","b"]},"labels":{"type":"array","items":{"type":"string"}}, + "ids":{"type":"array","items":{"type":"integer"}},"filter":{"type":"object"},"note":{"type":"string"}}} + """#) + let args = try f.arguments(from: [ + "startDate": "today", "limit": " 20 ", "ratio": "0.5", "dryRun": "true", "mode": "b", + "labels": "work, home ,", "ids": "1,2", "filter": #"{"a":[1]}"#, "note": " ", + ]).get() + #expect(args["startDate"] == .string("today")) + #expect(args["limit"] == .number(20)) + #expect(args["ratio"] == .number(0.5)) + #expect(args["dryRun"] == .bool(true)) + #expect(args["mode"] == .string("b")) + #expect(args["labels"] == .array([.string("work"), .string("home")])) + #expect(args["ids"] == .array([.number(1), .number(2)])) + #expect(args["filter"] == .object(["a": .array([.number(1)])])) + #expect(args["note"] == nil) // blank optional is omitted + } + + @Test func aBlankRequiredFieldIsAnError() { + let f = form(#"{"required":["q"],"properties":{"q":{"type":"string"}}}"#) + #expect(f.arguments(from: [:]) == .failure(.missingRequired("q"))) + #expect(f.arguments(from: ["q": " "]) == .failure(.missingRequired("q"))) + } + + @Test func badInputIsReportedPerField() { + let f = form(#"{"properties":{"n":{"type":"integer"},"x":{"type":"number"},"b":{"type":"boolean"},"m":{"enum":["a"]},"j":{"type":"object"},"ids":{"type":"array","items":{"type":"integer"}}}}"#) + func err(_ k: String, _ v: String) -> MCPToolInputError? { + if case .failure(let e) = f.arguments(from: [k: v]) { return e } + return nil + } + #expect(err("n", "1.5") == .invalid(field: "n", reason: "must be a whole number")) + #expect(err("x", "abc") != nil) + #expect(err("x", "nan") != nil) + #expect(err("b", "maybe") != nil) + #expect(err("m", "z") != nil) + #expect(err("j", "{oops") != nil) + #expect(err("ids", "1,two") != nil) + } + + @Test func aBooleanFieldNeverBlocksLaunchWhenBlank() throws { + let f = form(#"{"required":["flag"],"properties":{"flag":{"type":"boolean"}}}"#) + let args = try f.arguments(from: ["flag": ""]).get() + #expect(args["flag"] == .bool(false)) + } + + @Test func realTodoistScheduleSchemaYieldsAUsableForm() { + // Trimmed from Todoist's find-tasks-by-date inputSchema (fetched live 2026-09-20). + let f = form(#""" + {"type":"object","properties":{"cursor":{"type":"string"},"labels":{"type":"array","items":{"type":"string"}}, + "startDate":{"type":"string","pattern":"^(\\d{4}-\\d{2}-\\d{2}|today)$"},"daysCount":{"default":1,"minimum":1,"type":"integer","maximum":30}, + "limit":{"default":10,"type":"integer"},"overdueOption":{"type":"string","enum":["overdue-only","include-overdue","exclude-overdue"]}}} + """#) + #expect(!f.isEmpty) + #expect(!f.hasRequiredFields) + #expect(f.fields.first { $0.name == "overdueOption" }?.kind == .choice(["overdue-only", "include-overdue", "exclude-overdue"])) + #expect(f.fields.first { $0.name == "daysCount" }?.defaultText == "1") + } +} From e7e2e6d7b472eb061230a5b87da5119975f84f83 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:09:42 -0700 Subject: [PATCH 15/73] mcp-chat (2.0): MCP Apps in a chat with a local model Synced from locallmlab-sdk: Sources, packaging, README. A tool call whose tool links an MCP App shows the server's widget inline; the widget's own calls go through the session, so they meet the same per-server trust / approval as the model's. Qwen3-8B via MLX by default. Package.swift rewritten for the public tree: Core + Components from ../../Components, MCPAppsHost from ../../MCPAppsHost (both source), and its own Inference binaryTarget with the usual SDKRelease / knownSDKReleases / failManifest gate. Only 2.0.0-dev is known: the app does not build against 1.0. Three comments naming private-only examples or docs scrubbed. Verified: swift build against v2.0.0-dev; packaging/build-and-sign.sh (ad hoc, debug) embeds and signs both frameworks and the app launches. Also: README.md lists MCPAppsHost and mcp-chat; examples/README.md adds mcp-chat to the catalog and the "what each example needs" table; security-demo README (mirrored) links the SDK guide's 7c instead of a private doc, so check-public-hygiene.sh is clean. Co-Authored-By: Claude --- README.md | 6 + examples/README.md | 2 + examples/mcp-chat/Package.swift | 84 ++++ examples/mcp-chat/README.md | 85 ++++ .../mcp-chat/Sources/MCPChat/ChatModel.swift | 458 ++++++++++++++++++ .../mcp-chat/Sources/MCPChat/ChatViews.swift | 426 ++++++++++++++++ .../mcp-chat/Sources/MCPChat/MCPChatApp.swift | 42 ++ .../mcp-chat/Sources/MCPChat/WidgetSlot.swift | 211 ++++++++ examples/mcp-chat/packaging/Info.plist | 25 + .../mcp-chat/packaging/MCPChat.entitlements | 10 + examples/mcp-chat/packaging/build-and-sign.sh | 68 +++ examples/security-demo/README.md | 2 +- 12 files changed, 1418 insertions(+), 1 deletion(-) create mode 100644 examples/mcp-chat/Package.swift create mode 100644 examples/mcp-chat/README.md create mode 100644 examples/mcp-chat/Sources/MCPChat/ChatModel.swift create mode 100644 examples/mcp-chat/Sources/MCPChat/ChatViews.swift create mode 100644 examples/mcp-chat/Sources/MCPChat/MCPChatApp.swift create mode 100644 examples/mcp-chat/Sources/MCPChat/WidgetSlot.swift create mode 100644 examples/mcp-chat/packaging/Info.plist create mode 100644 examples/mcp-chat/packaging/MCPChat.entitlements create mode 100755 examples/mcp-chat/packaging/build-and-sign.sh diff --git a/README.md b/README.md index 9e371a3..5a4f801 100644 --- a/README.md +++ b/README.md @@ -81,6 +81,9 @@ what's public here is: [thisbrain.ai/locallm/cli.html](https://thisbrain.ai/locallm/cli.html). - **[Components/](Components/)** — `LocalLMLabSDKComponents`, prebuilt SwiftUI for managing MCP servers and models, built on the SDK's public API. +- **[MCPAppsHost/](MCPAppsHost/)** — `LocalLMLabSDKMCPAppsHost` (2.0), host-side support for + MCP Apps: the sandboxed view for a server's `ui://` widgets, the widget↔host bridge, and the + widget lifecycle. Source, built on the SDK's public API like `Components`. - **[skills/locallmlab-swift-app/](skills/locallmlab-swift-app/)** — a repo-local Agent Skill for building SDK apps with Claude Code or Codex. @@ -158,6 +161,9 @@ Roughly simplest to fullest — every one runnable, with full annotated source i - **[examples/vistanova/](examples/vistanova/)** — a tiny local search engine: web search through a Tavily MCP server, summaries from an MLX model **shipped pinned to an exact commit**, and defenses against a small model that skips the tool call. A SwiftUI app. +- **[examples/mcp-chat/](examples/mcp-chat/)** (2.0) — a chat with a local model where a tool call + that has an MCP App shows the server's interactive widget inline (Todoist's task list, a system + monitor); the widget's own calls go through the same per-server approval as the model's. Plus **[examples/api-lab/](examples/api-lab/)** (scripts + a chat app for the OpenAI-compatible endpoint) and **[examples/localai-cli/](examples/localai-cli/)** / **[localai-cli-swift/](examples/localai-cli-swift/)** diff --git a/examples/README.md b/examples/README.md index e47ac52..efa6316 100644 --- a/examples/README.md +++ b/examples/README.md @@ -22,6 +22,7 @@ Code samples for LocalLM Lab, split by which feature they use. | [security-demo/](security-demo/) | LocalLM Lab SDK (Core **+ Remote** + `Components`) | Tool authorization: a "Security" panel whose two controls are `Sequence.limited(toMaxImpact:)` (which tools the model sees) and `ConfirmingToolAuthorizer` + `Components.ToolConfirmationPresenter` (whether a call runs or asks first). A frontier model against a real Calendar and a real Todoist MCP server. | | [vistanova/](vistanova/) | LocalLM Lab SDK (Core **+ Inference**) | A tiny local search engine: a SwiftUI app that runs web search through a Tavily MCP server on one local model and summarizes on another — by default a downloaded MLX model **shipped pinned to an exact commit** (`MLXModelProvider(pinnedRevisions:)`), so a new upstream version is never picked up silently. Also shows a hand-written `Tool` over an MCP tool, and the defenses a small model needs: verify from `session.events` that it really called the tool, probe `capabilityProbe` before trusting structured output. | | [aiql/](aiql/) | LocalLM Lab SDK (Core **+ Inference**) | "Ask your data": a SwiftUI app that pulls an MCP-fronted dataset and writes the spreadsheet you asked for — `FileBackedTool` + the data verbs (`jsonToCsv` / `filterRows` / `sortRows` / …, guide §8b) do every row-level step so a local MLX model can't fabricate a value. Sandboxed; MCP OAuth via the `aiql://` URL scheme. | +| [mcp-chat/](mcp-chat/) | LocalLM Lab SDK (Core **+ Inference** + `Components` + `MCPAppsHost`), **2.0** | MCP Apps in a chat: a local MLX model (or Apple's on-device model) whose tool calls show the server's interactive widget inline in the conversation — Todoist's task list, a system monitor. The widget's own calls go through the session, so they meet the same per-server trust / approval as the model's. Streaming replies, reasoning split out, conversations that survive relaunch. Sandboxed `.app`. | **Path A vs Path B**, for the four Core-based examples above: two ways to turn a connector or MCP server into something the on-device model can call as a tool. **Path A** drops in a ready-made @@ -70,6 +71,7 @@ build against the latest stable release. No environment variable is needed for e | `mlx-control-room` | SwiftUI + App Sandbox (network only) | ✅ via the committed `.xcodeproj` — **Automatic** signing (as above); it keeps no security-scoped bookmark, so **Sign to Run Locally** works too | **any** identity — `packaging/build-and-sign.sh` | | `components-updates-demo` | SwiftUI, no system permissions, simulated data | ✅ via `Package.swift` / `swift run` (no `.xcodeproj`) | — (no `packaging/`) | | `workspace-buddy`, `workspace-buddy-local`, `aiql` | SwiftUI + App Sandbox | ✅ via the committed `.xcodeproj` — **Automatic** signing (as above); a stable team identity is what lets the security-scoped bookmark survive a rebuild. Bare `swift run` is compile-only. | same as `plate-today` | +| `mcp-chat` | SwiftUI + App Sandbox (network) | Compile-only via `Package.swift` / `swift build` (no `.xcodeproj`) | **any** identity, or **none** (`APP_IDENTITY=-`) — `packaging/build-and-sign.sh` | "Apple Development" = the free identity Xcode creates once you add any Apple ID under **Xcode ▸ Settings ▸ Accounts**. No paid Apple Developer account, no Developer ID certificate. diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift new file mode 100644 index 0000000..7d34bf6 --- /dev/null +++ b/examples/mcp-chat/Package.swift @@ -0,0 +1,84 @@ +// swift-tools-version: 6.4 +import Foundation +import PackageDescription + +// MCP Chat — a chat with a local model where a tool call that has an MCP App shows the server's +// interactive widget inline in the conversation. +// +// Qwen3-8B through MLX by default (Apple's on-device model as an alternative); MCP servers added +// with Components' server picker; widgets rendered by MCPAppsHost, routed back through the +// session so the user approves what a widget does exactly as they approve the model's calls. +// +// Inference ships as a binaryTarget (a GitHub Release asset). Core + Components come from the +// Components package (../../Components) and MCPAppsHost from ../../MCPAppsHost, both as source — +// this manifest must NOT declare its own LocalLMLabSDKCore binaryTarget: Components already +// declares that target, and two packages declaring a target of the same name is a hard SwiftPM +// error. Components re-vends Core as a `.library` product for exactly this. Same +// SDKRelease/knownSDKReleases/failManifest version gate as every other example here. +// +// Requires SDK 2.0.0-dev or later (MCP Apps support, per-server trust and approval are new in 2.0), +// macOS 27 + Xcode 27. No Metal Toolchain needed — the prebuilt Inference xcframework bundles the +// compiled default.metallib. + +struct SDKRelease { + let inferenceURL: String + let inferenceChecksum: String +} + +// Add an entry whenever a new Inference.xcframework release is published. The matching Core comes +// from Components' own knownSDKReleases table for the same LOCALLM_SDK_VERSION — keep the two +// in step. Only 2.0 releases: this app does not build against 1.0. +// The SDK release this example builds against with no setup. Build against another published +// version: set LOCALLM_SDK_VERSION in your shell (works for `swift build` / CI, NOT inside Xcode), +// or edit `defaultSDKVersion` here. +let defaultSDKVersion = "2.0.0-dev" + +let knownSDKReleases: [String: SDKRelease] = [ + "2.0.0-dev": SDKRelease( + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + ), +] + +func failManifest(_ message: String) -> Never { + FileHandle.standardError.write(Data((message + "\n").utf8)) + exit(1) +} + +let requested = ProcessInfo.processInfo.environment["LOCALLM_SDK_VERSION"] ?? defaultSDKVersion +guard let sdk = knownSDKReleases[requested] else { + failManifest(""" + error: Unknown LOCALLM_SDK_VERSION "\(requested)". + Known versions: \(knownSDKReleases.keys.sorted().joined(separator: ", ")) + """) +} + +let package = Package( + name: "MCPChat", + platforms: [.macOS("27.0")], + dependencies: [ + .package(path: "../../Components"), + .package(path: "../../MCPAppsHost"), + ], + targets: [ + .binaryTarget( + name: "LocalLMLabSDKInference", + url: sdk.inferenceURL, + checksum: sdk.inferenceChecksum + ), + .executableTarget( + name: "MCPChat", + dependencies: [ + .product(name: "LocalLMLabSDKCore", package: "Components"), + .product(name: "LocalLMLabSDKComponents", package: "Components"), + .product(name: "LocalLMLabSDKMCPAppsHost", package: "MCPAppsHost"), + "LocalLMLabSDKInference", + ], + linkerSettings: [ + // Bare SwiftPM executables get no LC_RPATH; SwiftPM extracts the xcframework + // slices next to the built binary. Same fix as code-buddy / model-switch. + .unsafeFlags(["-Xlinker", "-rpath", "-Xlinker", "@executable_path"]) + ] + ) + ] +) diff --git a/examples/mcp-chat/README.md b/examples/mcp-chat/README.md new file mode 100644 index 0000000..1ed88c3 --- /dev/null +++ b/examples/mcp-chat/README.md @@ -0,0 +1,85 @@ +# MCP Chat + +A chat with a local model where a tool call that has an MCP App shows the server's interactive +widget inline in the conversation. Ask "What's due today?" with Todoist +connected and the model calls `find-tasks-by-date`; the app sees that the tool links a widget and +shows Todoist's task list under the call. + +Qwen3 8B through MLX by default; Apple's on-device model as an alternative (Settings). + +`Package.swift` builds against SDK `2.0.0-dev` with no further setup (MCP Apps support is new in +2.0): `LocalLMLabSDKCore` and `LocalLMLabSDKInference` (the MLX runtime) come from that GitHub +Release, `Components` (`../../Components`) and `MCPAppsHost` (`../../MCPAppsHost`) from this repo as +source. Needs macOS 27 and Xcode 27 (Swift 6.4). No Metal Toolchain needed — the prebuilt Inference +xcframework bundles the compiled shaders. + +## Build and run + +```bash +packaging/build-and-sign.sh # sandboxed release build → dist/MCP Chat.app +open "dist/MCP Chat.app" +``` + +The first launch offers to download Qwen3 8B (about 4.6 GB, pinned to a reviewed revision) into the +app's sandbox container. + +For development, a build without App Sandbox can use a model you already have instead: + +```bash +CONFIG=debug SANDBOX=0 packaging/build-and-sign.sh +open --env MCPCHAT_MODEL_CACHE=/path/to/huggingface/hub "dist/MCP Chat.app" +``` + +## Try it + +1. Add a server with the server button in the toolbar: + - System Monitor (`examples/system-monitor-server` in + [modelcontextprotocol/ext-apps](https://github.com/modelcontextprotocol/ext-apps), run locally): + `http://localhost:3001/mcp`, auth **None**. + - Todoist: `https://ai.todoist.net/mcp`, auth **None** (OAuth with dynamic registration: you sign + in in your browser when you click Add). +2. **Turn tools on.** A new server's tools start off (the SDK's default — a server's tools reach the + model only when the user opts in). The app shows a banner until at least one is on. For + Todoist: `find-tasks-by-date` (the one with a widget), `add-tasks`, `complete-tasks`, + `reschedule-tasks`, `find-projects`. For System Monitor: `get-system-info`. +3. Ask "How is my system doing?" or "What's due tomorrow?". + +## What it shows + +- **Widgets from tool calls.** The model only chooses tools. A tool whose listing carries + `_meta.ui.resourceUri` gets its widget drawn under the call when it finishes, fed the call's + arguments and result. The SDK adds one line to such tools' descriptions + (`makeSession(…, mcpAppHints: true)`) so a model can prefer them when asked to *see* something. +- **One gate for model and widget.** A widget's own tool calls go through the session + (`MCPAppsSessionBackend`), so they meet the same approval as the model's and are recorded + ("by a view"). Each server has a trust level and a tool approval, set on its row in the server + list (Components' picker): an untrusted server (the default) asks before anything that may + change data; a trusted one has its read-only / destructive labels believed and asks only before + destructive tools. `ConfirmingToolAuthorizer` remembers a widget's answer per tool, so a polling + dashboard asks once, not every two seconds. +- **Live widget limit.** At most 3 widgets stay live (about 100 MB each); older ones become + snapshots until scrolled back to or clicked, then are re-created from the cache, checked against + the server (`MCPAppRecreation`): newer compatible version → shown with the stored result; output + changed shape → a Refresh button; offline → view-only. +- **Streaming.** `session.streamResponse(to:)`: the user's message shows at once, the reply streams + (`hostTranscript.replyInProgress`). The SDK splits Qwen3's inline `` block into a + `.reasoning` entry, shown as a collapsed Reasoning section. +- **Date and time.** The current date and time go to the model with every message + (`session.turnContext`), and the SDK's `ClockTool` is on by default (Settings → Tools), so + "tomorrow" means tomorrow. +- **Conversations survive relaunch.** What the user saw is saved with `hostTranscript.archive()`. + With "Remember conversations" on (default), the model's transcript is saved too and the next + launch continues with it (`makeSession(…, restoring:)`); off, the user still sees the old + conversation, the model starts fresh, and a divider says so. +- **Tools changing mid-chat.** A session follows `lab.mcp`: a tool turned on, a server added, or a + trust change reaches the model on its next turn, in the same session. Switching the model (or + the app's own tools) moves the conversation to a new session, model memory and host transcript + both carried over. + +## Files + +| File | | +|---|---| +| `ChatModel.swift` | lab, servers, sessions (new one on a model switch), approvals, save/restore, settings | +| `WidgetSlot.swift` | a call's widget: fetch/cache, re-create, refresh, bridge policy | +| `ChatViews.swift` | conversation, composer, tools badge, approval toggles, settings | diff --git a/examples/mcp-chat/Sources/MCPChat/ChatModel.swift b/examples/mcp-chat/Sources/MCPChat/ChatModel.swift new file mode 100644 index 0000000..fcce48e --- /dev/null +++ b/examples/mcp-chat/Sources/MCPChat/ChatModel.swift @@ -0,0 +1,458 @@ +import AppKit +import Combine +import Foundation +import FoundationModels +import LocalLMLabSDKComponents +import LocalLMLabSDKCore +import LocalLMLabSDKInference +import LocalLMLabSDKMCPAppsHost + +// The model this app routes to by default, pinned to the commit it was tried against (see +// workspace-buddy-local for why a revision is pinned). +let qwenRepo = "mlx-community/Qwen3-8B-4bit" +let qwenRevision = "545dc4251c05440727734bcd94334791f6ab0192" + +extension RouteName { + static let chat: RouteName = "chat" +} + +enum ChatModelChoice: String, CaseIterable, Identifiable { + case qwen3 + case apple + + var id: String { rawValue } + var title: String { + switch self { + case .qwen3: "Qwen3 8B (MLX, local)" + case .apple: "Apple on-device model" + } + } + var modelID: ModelID { + switch self { + case .qwen3: ModelID(scheme: "mlx", rest: qwenRepo)! + case .apple: .system + } + } +} + +/// App settings (Settings window). Read where they are used, so a change applies from the next +/// turn without restarting. +enum ChatSettings { + static let modelKey = "model" + /// Whether a reopened conversation continues with the model's memory of it. Off: the user still + /// sees the earlier conversation, but the model starts fresh (and nothing the model saw is + /// written to disk). + static let rememberKey = "rememberConversation" + static let widgetMessagesKey = "allowWidgetMessages" + static let widgetContextKey = "allowWidgetContext" + static let liveWidgetsKey = "liveWidgetLimit" + static let clockToolKey = "clockTool" + + static func register() { + UserDefaults.standard.register(defaults: [ + modelKey: ChatModelChoice.qwen3.rawValue, + rememberKey: true, + widgetMessagesKey: false, + widgetContextKey: true, + liveWidgetsKey: 3, + clockToolKey: true, + ]) + } + + static var model: ChatModelChoice { + ChatModelChoice(rawValue: UserDefaults.standard.string(forKey: modelKey) ?? "") ?? .qwen3 + } + static var remember: Bool { UserDefaults.standard.bool(forKey: rememberKey) } + static var allowWidgetMessages: Bool { UserDefaults.standard.bool(forKey: widgetMessagesKey) } + static var allowWidgetContext: Bool { UserDefaults.standard.bool(forKey: widgetContextKey) } + static var liveWidgets: Int { max(1, UserDefaults.standard.integer(forKey: liveWidgetsKey)) } + static var clockTool: Bool { UserDefaults.standard.bool(forKey: clockToolKey) } +} + +@MainActor +final class ChatModel: ObservableObject { + enum Status: Equatable { + /// The chosen model is not ready (e.g. Qwen3 is not downloaded yet). + case needsModel + case downloading(Double) + case ready + case responding + } + + let manager: MCPServerManager + let servers: MCPServerManagerObservable + let toolConfirm = ToolConfirmationPresenter() + let widgetCache = MCPAppWidgetCache() + let pool = MCPAppViewPool(limit: ChatSettings.liveWidgets) + + @Published private(set) var session: LocalLMLabSession? + @Published private(set) var status: Status = .needsModel + @Published private(set) var modelChoice = ChatSettings.model + /// Set when a conversation was reopened without the model's memory: turns up to this one were + /// not given to the model. + @Published private(set) var memoryBoundaryTurn: Int? + @Published var errorMessage: String? + @Published var showServers = false + /// One authorizer for every session. It follows each server's trust and tool approval, which + /// the user sets per server in the server list (Components' picker), and remembers a widget's + /// answers; a session made after a model switch keeps them. + private let authorizer: ConfirmingToolAuthorizer + + // Development only: MCPCHAT_MODEL_CACHE points at an existing Hugging Face cache, so a build + // without App Sandbox (SANDBOX=0 packaging/build-and-sign.sh) can skip the download. A + // sandboxed build can't read outside its container and downloads into it. + private let mlx = MLXModelProvider( + cacheDirectory: ProcessInfo.processInfo.environment["MCPCHAT_MODEL_CACHE"].map { URL(fileURLWithPath: $0) }, + residentModelLimit: 1, pinnedRevisions: [qwenRepo: qwenRevision]) + private let lab: LocalLMLab + private let store = ConversationStore() + private var sessionSignature: String? + private var cancellables: Set = [] + + static let instructions = """ + You are a helpful assistant in a chat app. Use the tools you have when the user asks about \ + their data or asks you to change it. Some tools also show the user an interactive view of \ + the result; when one does, keep your reply short and do not repeat what the view shows. + """ + + init() { + ChatSettings.register() + // The lab makes the one MCP manager (`lab.mcp`). Declare MCP Apps support so servers that + // gate their widgets on it offer them. + let lab = LocalLMLab(configuration: .init( + providers: [mlx, SystemModelProvider()], + mcp: MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps()))) + self.lab = lab + self.manager = lab.mcp + self.servers = MCPServerManagerObservable(core: lab.mcp) + authorizer = ConfirmingToolAuthorizer(channel: toolConfirm) + modelChoice = ChatSettings.model + lab.models.route(.chat, to: modelChoice.modelID) + } + + // MARK: launch + + func start() async { + await restoreServers() + refreshModelStatus() + if status == .ready { openSavedConversation() } + + NotificationCenter.default.publisher(for: UserDefaults.didChangeNotification) + .receive(on: RunLoop.main) + .sink { [weak self] _ in self?.settingsChanged() } + .store(in: &cancellables) + NotificationCenter.default.publisher(for: NSApplication.willTerminateNotification) + .sink { [weak self] _ in MainActor.assumeIsolated { self?.save() } } + .store(in: &cancellables) + } + + private func refreshModelStatus() { + switch lab.models.availability(for: modelChoice.modelID) { + case .available: if status != .responding { status = .ready } + default: status = .needsModel + } + } + + var modelUnavailableReason: String? { + switch lab.models.availability(for: modelChoice.modelID) { + case .available, .notDownloaded: nil + case .needsCredential: "This model needs a credential." + case .unavailable(_, let detail): detail + @unknown default: "This model is not available." + } + } + + func downloadModel() async { + guard modelChoice == .qwen3, status == .needsModel else { return } + status = .downloading(0) + do { + if let preflight = try? await mlx.validate(qwenRepo), !preflight.passed { + throw ChatError("This Mac can't run \(qwenRepo): \(preflight.detail ?? "pre-flight failed").") + } + for try await event in mlx.download(qwenRepo) { + if case .progress(_, _, let fraction) = event { status = .downloading(fraction) } + } + } catch { + errorMessage = "Download failed: \(error.localizedDescription)" + } + status = .needsModel + refreshModelStatus() + if status == .ready, session == nil { openSavedConversation() } + } + + private func settingsChanged() { + pool.limit = ChatSettings.liveWidgets + if !ChatSettings.remember { store.forgetModelMemory() } + guard ChatSettings.model != modelChoice, status != .responding else { return } + modelChoice = ChatSettings.model + lab.models.route(.chat, to: modelChoice.modelID) + refreshModelStatus() + // The session is rebuilt on the next turn, carrying the conversation over to the new model. + if status == .ready, session == nil { openSavedConversation() } + } + + // MARK: conversation + + /// Reopens the saved conversation: always what the user saw; the model's memory of it only + /// when "Remember conversations" is on. + private func openSavedConversation() { + let saved = store.load() + let memory = ChatSettings.remember ? saved?.model : nil + do { + let session = try makeSession(restoring: memory) + if let shown = saved?.shown { try session.hostTranscript.restore(from: shown) } + if memory == nil, let last = session.hostTranscript.entries.last?.turn { memoryBoundaryTurn = last } + install(session) + } catch { + errorMessage = "Couldn't open the saved conversation: \(error.localizedDescription)" + if let fresh = try? makeSession(restoring: nil) { install(fresh) } + } + } + + func newConversation() { + guard status != .responding else { return } + Task { await pool.releaseAll() } + store.clear() + memoryBoundaryTurn = nil + session = nil + sessionSignature = nil + if status == .ready, let fresh = try? makeSession(restoring: nil) { install(fresh) } + } + + func send(_ text: String) { + let prompt = text.trimmingCharacters(in: .whitespacesAndNewlines) + guard !prompt.isEmpty else { return } + Task { await respond { $0.streamResponse(to: prompt) } } + } + + /// A widget's `ui/message` — only reached when Settings allows widget messages. + func sendFromWidget(_ text: String, instance: String) { + Task { await respond { $0.streamResponse(to: text, fromAppInstance: instance) } } + } + + /// Runs a streamed turn. The view draws the reply as it arrives from + /// `hostTranscript.replyInProgress`, so the stream only needs draining here. + private func respond(_ turn: (LocalLMLabSession) -> AsyncThrowingStream) async { + guard status == .ready else { return } + let session: LocalLMLabSession + do { + session = try currentSession() + } catch { + errorMessage = error.localizedDescription + return + } + status = .responding + defer { + status = .ready + save() + } + do { + for try await _ in turn(session) {} + } catch { + errorMessage = await GenerationErrorDescription.describe(error) + } + } + + /// The session for the next turn. A session follows `lab.mcp` by itself (tools turned on or + /// off, servers added, trust changed: picked up on its next turn). What it can't follow is a + /// different model or a change to the app's own tools; then a new session takes over the + /// conversation: the model's memory (its transcript) and what the user sees (the archive). + private func currentSession() throws -> LocalLMLabSession { + if let session, sessionSignature == signature() { return session } + guard let old = session else { + let fresh = try makeSession(restoring: nil) + install(fresh) + return fresh + } + let fresh = try makeSession(restoring: old.languageModelSession.transcript) + try fresh.hostTranscript.restore(from: old.hostTranscript.archive()) + // Live widgets call through the session they were made with: re-create them on the new one. + Task { await pool.releaseAll() } + install(fresh) + return fresh + } + + private func makeSession(restoring memory: Transcript?) throws -> LocalLMLabSession { + // A saved conversation carries its own instructions; a new one gets the app's. + var session = try lab.makeSession( + route: .chat, tools: builtInTools, instructions: memory == nil ? Self.instructions : nil, + restoring: memory, mcpAppHints: true, authorizer: authorizer) + session.retryOnContextOverflow = RetryPolicy(maxRetries: 2, compact: Self.dropOldestHalf) + session.turnContext = { Self.clock(Date()) } + sessionSignature = signature() + return session + } + + /// The app's own tools, alongside the MCP servers'. The SDK's clock (`getCurrentTime`, rated + /// read-only, so it never asks) lets the model check the date when it needs to — e.g. to turn + /// "tomorrow" into a date for Todoist. It is in addition to the date sent with each turn. + private var builtInTools: [any Tool] { + ChatSettings.clockTool ? [ClockTool()] : [] + } + + var builtInToolNames: [String] { builtInTools.map(\.name) } + + private func install(_ session: LocalLMLabSession) { + self.session = session + } + + /// What a session can't change by itself: the model and the app's own tools. + private func signature() -> String { + ([modelChoice.rawValue] + builtInToolNames).joined(separator: "\n") + } + + /// The current date and time, sent with every turn: a model has no clock, so without it + /// "what's due tomorrow?" resolves against a date from its training data. + nonisolated static func clock(_ now: Date, timeZone: TimeZone = .current) -> String { + var calendar = Calendar(identifier: .gregorian) + calendar.timeZone = timeZone + let parts = calendar.dateComponents([.year, .month, .day, .hour, .minute, .weekday], from: now) + let weekday = calendar.weekdaySymbols[(parts.weekday ?? 1) - 1] + let offset = timeZone.secondsFromGMT(for: now) / 60 + let date = String(format: "%04d-%02d-%02d", parts.year ?? 0, parts.month ?? 0, parts.day ?? 0) + let time = String(format: "%02d:%02d", parts.hour ?? 0, parts.minute ?? 0) + let utc = String(format: "UTC%@%02d:%02d", offset < 0 ? "-" : "+", abs(offset) / 60, abs(offset) % 60) + return "[Current date and time: \(weekday) \(date) \(time), \(timeZone.identifier) (\(utc)). " + + "Use it for relative dates such as today, tomorrow or next week.]" + } + + /// Context overflow (e.g. a long conversation moved to the smaller on-device model): keep the + /// instructions and the newer half of the conversation, starting at a user turn. + nonisolated static func dropOldestHalf(_ transcript: Transcript) -> Transcript { + let entries = Array(transcript) + let prompts = entries.indices.filter { if case .prompt = entries[$0] { true } else { false } } + guard prompts.count > 1 else { return transcript } + let keepFrom = prompts[prompts.count / 2] + var kept: [Transcript.Entry] = [] + if let first = entries.first, case .instructions = first { kept.append(first) } + kept.append(contentsOf: entries[keepFrom...]) + return Transcript(entries: kept) + } + + private func save() { + guard let session else { return } + store.save(shown: try? session.hostTranscript.archive(), + model: ChatSettings.remember ? session.languageModelSession.transcript : nil) + } + + // MARK: servers (saved list; quiet reconnect) + + private struct SavedServer: Codable { + var url: URL + var name: String + var authType: MCPAuthType + var manualClientID: String? + var noAuth: Bool + /// The server's tools as last seen, with the user's on/off choice for each. New tools from + /// a server start off (the SDK's default); restoring these keeps the user's choices. + var tools: [MCPToolDescriptor]? + /// The user's trust and tool approval for the server (server list). The manager doesn't + /// persist them; they are set again after restore. + var trust: MCPServerTrust? + var toolApproval: ToolApproval? + } + + private let savedServersKey = "servers.v1" + + private func restoreServers() async { + let saved = (UserDefaults.standard.data(forKey: savedServersKey) + .flatMap { try? JSONDecoder().decode([SavedServer].self, from: $0) }) ?? [] + manager.restore(from: saved.map { + (id: MCPServerID(rawValue: $0.url.absoluteString), url: $0.url, displayName: $0.name, tools: $0.tools ?? [], estimatedTokens: 0, + enabled: true, authType: $0.authType, manualClientID: $0.manualClientID, resources: []) + }) + // Trust before reconnecting: trusting records the saved tool list, so a server whose tools + // changed while the app was closed shows "tools changed since trusted". + for server in saved { + let id = MCPServerID(rawValue: server.url.absoluteString) + if let trust = server.trust { manager.setTrust(trust, server: id) } + manager.setToolApproval(server.toolApproval, server: id) + } + // Only servers that need no sign-in, or have a stored credential, reconnect at launch: this + // app never opens a browser unprompted. + for server in saved where server.noAuth || MCPCredentialProbe.storedKind(for: server.url) != nil { + _ = await manager.reconnect(MCPServerID(rawValue: server.url.absoluteString)) + } + servers.$servers.dropFirst().sink { [weak self] in self?.persist($0) }.store(in: &cancellables) + } + + private func persist(_ states: [MCPServerID: MCPServerState]) { + let previous = (UserDefaults.standard.data(forKey: savedServersKey) + .flatMap { try? JSONDecoder().decode([SavedServer].self, from: $0) }) ?? [] + let saved = states.values.map { state in + SavedServer(url: state.url, name: state.displayName, authType: state.authType, manualClientID: state.manualClientID, + noAuth: state.connectionStatus == .connected + ? MCPCredentialProbe.storedKind(for: state.url) == nil + : previous.first { $0.url == state.url }?.noAuth ?? false, + tools: state.tools.isEmpty ? previous.first { $0.url == state.url }?.tools : state.tools, + trust: state.trust, toolApproval: state.toolApproval) + } + if let data = try? JSONEncoder().encode(saved) { UserDefaults.standard.set(data, forKey: savedServersKey) } + } + + /// The MCP tools the model is given on its next turn. + var modelTools: [MCPToolDescriptor] { manager.toolsForSession() } + + /// Connected servers none of whose tools are on: the model can't use them yet. (New tools start + /// off; the user turns on the ones they want in the server list.) + var serversWithNoToolsOn: [MCPServerState] { + servers.sortedServers.filter { state in + state.connectionStatus == .connected && !state.tools.isEmpty + && !state.tools.contains { $0.enabled && $0.isModelVisible } + } + } + + /// The server a tool call went to, for its widget. + func serverID(of record: ToolCallRecord) -> MCPServerID? { + if case .mcp(let server, _) = record.tool { return server } + return nil + } +} + +struct ChatError: LocalizedError { + let errorDescription: String? + init(_ message: String) { errorDescription = message } +} + +/// The open conversation on disk (Application Support/MCPChat): what the user saw +/// (`HostTranscript.archive()`) and, when "Remember conversations" is on, the model's transcript. +struct ConversationStore { + struct Saved { + var shown: Data? + var model: Transcript? + } + + private let directory: URL = { + let base = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask)[0] + .appendingPathComponent("MCPChat/Conversation", isDirectory: true) + try? FileManager.default.createDirectory(at: base, withIntermediateDirectories: true) + return base + }() + + private var shownFile: URL { directory.appendingPathComponent("shown.json") } + private var modelFile: URL { directory.appendingPathComponent("model.json") } + + func load() -> Saved? { + let shown = try? Data(contentsOf: shownFile) + let model = (try? Data(contentsOf: modelFile)).flatMap { try? JSONDecoder().decode(Transcript.self, from: $0) } + return shown == nil && model == nil ? nil : Saved(shown: shown, model: model) + } + + func save(shown: Data?, model: Transcript?) { + if let shown { try? shown.write(to: shownFile, options: .atomic) } + if let model, let data = try? JSONEncoder().encode(model) { + try? data.write(to: modelFile, options: .atomic) + } else { + forgetModelMemory() + } + } + + func forgetModelMemory() { + try? FileManager.default.removeItem(at: modelFile) + } + + func clear() { + try? FileManager.default.removeItem(at: shownFile) + forgetModelMemory() + } +} diff --git a/examples/mcp-chat/Sources/MCPChat/ChatViews.swift b/examples/mcp-chat/Sources/MCPChat/ChatViews.swift new file mode 100644 index 0000000..97c8167 --- /dev/null +++ b/examples/mcp-chat/Sources/MCPChat/ChatViews.swift @@ -0,0 +1,426 @@ +import LocalLMLabSDKComponents +import LocalLMLabSDKCore +import LocalLMLabSDKMCPAppsHost +import SwiftUI + +struct ChatRootView: View { + @ObservedObject var chat: ChatModel + + var body: some View { + VStack(spacing: 0) { + if let session = chat.session { + ConversationView(chat: chat, transcript: session.hostTranscript) + } else { + ModelSetupView(chat: chat) + } + ForEach(chat.serversWithNoToolsOn, id: \.id) { server in + HStack(spacing: 8) { + Image(systemName: "exclamationmark.circle").foregroundStyle(.orange) + Text("\(server.displayName) is connected, but none of its tools are on, so the model can't use it.") + Spacer() + Button("Choose Tools…") { chat.showServers = true } + } + .font(.callout) + .padding(.horizontal, 12).padding(.vertical, 8) + .background(.orange.opacity(0.1)) + } + Divider() + Composer(chat: chat) + } + .frame(minWidth: 640, minHeight: 560) + .toolbar { + ToolbarItem(placement: .navigation) { + Text(chat.modelChoice.title).font(.callout).foregroundStyle(.secondary) + } + ToolbarItemGroup { + ToolsBadge(chat: chat) + Button { chat.showServers = true } label: { Label("MCP Servers", systemImage: "server.rack") } + Button { chat.newConversation() } label: { Label("New Chat", systemImage: "square.and.pencil") } + .disabled(chat.status == .responding) + } + } + .sheet(isPresented: $chat.showServers) { + VStack(alignment: .trailing) { + MCPServerPickerView(manager: chat.servers) + Button("Done") { chat.showServers = false }.keyboardShortcut(.defaultAction) + } + .padding() + .frame(minWidth: 560, minHeight: 480) + } + .toolConfirmationSheet(chat.toolConfirm) + .alert("Something went wrong", isPresented: Binding(get: { chat.errorMessage != nil }, set: { if !$0 { chat.errorMessage = nil } })) { + Button("OK") { chat.errorMessage = nil } + } message: { + Text(chat.errorMessage ?? "") + } + .task { await chat.start() } + } +} + +private struct ModelSetupView: View { + @ObservedObject var chat: ChatModel + + var body: some View { + VStack(spacing: 14) { + Image(systemName: "bubble.left.and.text.bubble.right").font(.system(size: 40)).foregroundStyle(.tint) + Text("MCP Chat").font(.title.weight(.semibold)) + switch chat.status { + case .downloading(let fraction): + ProgressView(value: fraction) { Text("Downloading \(qwenRepo) — \(Int(fraction * 100))%") } + .frame(maxWidth: 360) + default: + if let reason = chat.modelUnavailableReason { + Text(reason).foregroundStyle(.secondary).multilineTextAlignment(.center).frame(maxWidth: 380) + Text("Choose another model in Settings.").font(.callout).foregroundStyle(.secondary) + } else if chat.modelChoice == .qwen3 { + Text("Qwen3 8B runs on this Mac. It needs a one-time download of about 4.6 GB.") + .foregroundStyle(.secondary).multilineTextAlignment(.center).frame(maxWidth: 380) + Button("Download Model") { Task { await chat.downloadModel() } } + .buttonStyle(.borderedProminent).controlSize(.large) + } + } + Text("Add MCP servers with the server button in the toolbar. When a tool has an interactive view, it appears in the conversation.") + .font(.footnote).foregroundStyle(.secondary).multilineTextAlignment(.center).frame(maxWidth: 380) + } + .padding(40).frame(maxWidth: .infinity, maxHeight: .infinity) + } +} + +private struct ConversationView: View { + @ObservedObject var chat: ChatModel + let transcript: HostTranscript + @State private var position = ScrollPosition(edge: .bottom) + + var body: some View { + ScrollView { + LazyVStack(alignment: .leading, spacing: 12) { + if transcript.entries.isEmpty { + Text("Ask something. Try “What's due today?” with Todoist connected.") + .foregroundStyle(.secondary).frame(maxWidth: .infinity).padding(.top, 60) + } + ForEach(transcript.entries) { entry in + EntryView(entry: entry, chat: chat) + .id(entry.id) + if let boundary = chat.memoryBoundaryTurn, entry.id == lastID(ofTurn: boundary) { + MemoryDivider() + } + } + if let reply = transcript.replyInProgress { + ReplyInProgressView(reply: reply) + } else if chat.status == .responding { + HStack(spacing: 8) { ProgressView().controlSize(.small); Text("Thinking…").foregroundStyle(.secondary) } + } + } + .padding(.horizontal, 16) + .padding(.top, 16) + // Room below the newest item, so it never sits against the message box. + .padding(.bottom, 72) + } + .scrollPosition($position) + .defaultScrollAnchor(.bottom) + // Follow the conversation while the user is at the bottom: whatever makes it taller — a new + // message, the spinner, streaming text, a widget loading or resizing — scrolls it into view. + // Someone who scrolled up to read stays where they are. + .onScrollGeometryChange(for: Layout.self) { geometry in + Layout(contentHeight: geometry.contentSize.height, + distanceFromBottom: geometry.contentSize.height - geometry.contentOffset.y - geometry.containerSize.height) + } action: { old, new in + if new.contentHeight != old.contentHeight, old.distanceFromBottom < 120 { + position.scrollTo(edge: .bottom) + } + } + // Sending a message always brings the bottom into view. + .onChange(of: chat.status) { _, status in + if status == .responding { withAnimation { position.scrollTo(edge: .bottom) } } + } + } + + private struct Layout: Equatable { + var contentHeight: CGFloat + var distanceFromBottom: CGFloat + } + + private func lastID(ofTurn turn: Int) -> String? { + transcript.entries.last { $0.turn == turn }?.id + } +} + +private struct MemoryDivider: View { + var body: some View { + HStack { + VStack { Divider() } + Text("The model doesn't remember the messages above").font(.caption).foregroundStyle(.secondary).fixedSize() + VStack { Divider() } + } + .help("“Remember conversations” was off when this conversation was saved or reopened. Turn it on in Settings to keep the model's memory across relaunches.") + } +} + +private struct EntryView: View { + let entry: HostTranscript.Entry + @ObservedObject var chat: ChatModel + + var body: some View { + switch entry.content { + case .user(let text): + UserBubble(text: text, fromWidget: entry.appInstance != nil) + case .assistant(let text): + AssistantMessage(text: text) + case .reasoning(let text): + ReasoningDisclosure(text: text) + case .toolCall(let record): + ToolCallView(record: record, chat: chat) + case .appContext(_, let text): + DisclosureGroup { + Text(text).font(.caption.monospaced()).textSelection(.enabled).frame(maxWidth: .infinity, alignment: .leading) + } label: { + Label("A view shared context with the model", systemImage: "rectangle.and.text.magnifyingglass") + .font(.caption).foregroundStyle(.secondary) + } + @unknown default: + EmptyView() + } + } +} + +private struct UserBubble: View { + let text: String + let fromWidget: Bool + + var body: some View { + HStack { + Spacer(minLength: 80) + VStack(alignment: .trailing, spacing: 3) { + if fromWidget { + Label("Sent by a view", systemImage: "rectangle.on.rectangle").font(.caption2).foregroundStyle(.secondary) + } + Text(text) + .textSelection(.enabled) + .padding(.horizontal, 12).padding(.vertical, 8) + .background(Color.accentColor.opacity(fromWidget ? 0.08 : 0.15), in: RoundedRectangle(cornerRadius: 12)) + } + } + } +} + +/// The answer. Reasoning arrives as its own `.reasoning` entry: the SDK splits Qwen3's inline +/// `` block out of the reply. +private struct AssistantMessage: View { + let text: String + + var body: some View { + Text(Self.markdown(text)) + .textSelection(.enabled) + .frame(maxWidth: .infinity, alignment: .leading) + } + + static func markdown(_ text: String) -> AttributedString { + (try? AttributedString(markdown: text, options: .init(interpretedSyntax: .inlineOnlyPreservingWhitespace))) + ?? AttributedString(text) + } +} + +/// The streaming reply (`hostTranscript.replyInProgress`, reasoning already split out). +private struct ReplyInProgressView: View { + let reply: HostTranscript.ReplyInProgress + + var body: some View { + VStack(alignment: .leading, spacing: 6) { + if let reasoning = reply.reasoning { ReasoningDisclosure(text: reasoning, thinking: reply.isReasoning) } + if reply.text.isEmpty { + if reply.reasoning == nil { + HStack(spacing: 8) { ProgressView().controlSize(.small); Text("Thinking…").foregroundStyle(.secondary) } + } + } else { + AssistantMessage(text: reply.text) + } + } + } +} + +/// The model's reasoning, collapsed by default. +private struct ReasoningDisclosure: View { + let text: String + var thinking = false + + var body: some View { + DisclosureGroup { + Text(text).font(.callout).foregroundStyle(.secondary).textSelection(.enabled) + .frame(maxWidth: .infinity, alignment: .leading) + } label: { + HStack(spacing: 6) { + Label(thinking ? "Thinking…" : "Reasoning", systemImage: "brain").font(.caption).foregroundStyle(.secondary) + if thinking { ProgressView().controlSize(.mini) } + } + } + } +} + +private struct ToolCallView: View { + let record: ToolCallRecord + @ObservedObject var chat: ChatModel + + var body: some View { + VStack(alignment: .leading, spacing: 6) { + DisclosureGroup { + VStack(alignment: .leading, spacing: 4) { + Text("Arguments").font(.caption2).foregroundStyle(.secondary) + Text(record.argumentsDescription).font(.caption.monospaced()).textSelection(.enabled) + if let output = record.output { + Text("Result").font(.caption2).foregroundStyle(.secondary) + Text(output.prefix(2000)).font(.caption.monospaced()).textSelection(.enabled) + } + } + .frame(maxWidth: .infinity, alignment: .leading) + } label: { + HStack(spacing: 6) { + stateIcon + Text(record.tool.name).font(.callout.monospaced()) + if let by = initiatorLabel { Text(by).font(.caption2).foregroundStyle(.secondary) } + if case .denied(let reason) = record.state { Text("Not allowed: \(reason)").font(.caption).foregroundStyle(.secondary) } + if case .failed(let reason) = record.state { Text(reason).font(.caption).foregroundStyle(.red).lineLimit(1) } + } + } + if record.app != nil, record.state == .finished, record.mcpResult.map({ !$0.isError }) ?? false { + WidgetSlot(record: record, chat: chat, pool: chat.pool) + } + } + } + + private var initiatorLabel: String? { + switch record.initiator { + case .model: nil + case .app: "by a view" + case .host: "refresh" + @unknown default: nil + } + } + + @ViewBuilder private var stateIcon: some View { + switch record.state { + case .awaitingApproval: Image(systemName: "hand.raised").foregroundStyle(.orange) + case .running: ProgressView().controlSize(.mini) + case .finished: Image(systemName: "checkmark.circle").foregroundStyle(.green) + case .failed: Image(systemName: "exclamationmark.triangle").foregroundStyle(.red) + case .denied: Image(systemName: "nosign").foregroundStyle(.secondary) + @unknown default: Image(systemName: "questionmark.circle") + } + } +} + +struct Composer: View { + @ObservedObject var chat: ChatModel + @State private var text = "" + @FocusState private var focused: Bool + + var body: some View { + HStack(alignment: .bottom, spacing: 8) { + TextField("Message", text: $text, axis: .vertical) + .lineLimit(1...6) + .textFieldStyle(.plain) + .focused($focused) + .onSubmit(send) + Button(action: send) { Image(systemName: "arrow.up.circle.fill").font(.title2) } + .buttonStyle(.plain) + .disabled(!canSend) + .keyboardShortcut(.return, modifiers: .command) + } + .padding(12) + .onAppear { focused = true } + } + + private var canSend: Bool { + chat.status == .ready && !text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + } + + private func send() { + guard canSend else { return } + chat.send(text) + text = "" + } +} + +struct SettingsView: View { + @AppStorage(ChatSettings.modelKey) private var model = ChatModelChoice.qwen3.rawValue + @AppStorage(ChatSettings.rememberKey) private var remember = true + @AppStorage(ChatSettings.widgetMessagesKey) private var widgetMessages = false + @AppStorage(ChatSettings.widgetContextKey) private var widgetContext = true + @AppStorage(ChatSettings.liveWidgetsKey) private var liveWidgets = 3 + @AppStorage(ChatSettings.clockToolKey) private var clockTool = true + + var body: some View { + Form { + Section("Model") { + Picker("Model", selection: $model) { + ForEach(ChatModelChoice.allCases) { Text($0.title).tag($0.rawValue) } + } + Toggle("Remember conversations", isOn: $remember) + Text("When on, a reopened conversation continues with the model's memory of it, and what the model saw — including tool results — is saved on this Mac. When off, you still see the earlier conversation, but the model starts fresh.") + .font(.caption).foregroundStyle(.secondary) + } + Section("Tools") { + Toggle("Clock tool (getCurrentTime)", isOn: $clockTool) + Text("Lets the model check the current date and time, e.g. for “what's due tomorrow?”. The date is also sent with every message.") + .font(.caption).foregroundStyle(.secondary) + Text("Whether a server's tools ask before they run is set per server in the server list: its trust level and tool approval.") + .font(.caption).foregroundStyle(.secondary) + } + Section("Interactive views") { + Toggle("Views may share context with the model", isOn: $widgetContext) + Toggle("Views may send messages as you", isOn: $widgetMessages) + Stepper("Live views: \(liveWidgets)", value: $liveWidgets, in: 1...8) + Text("Each live view uses about 100 MB. Older views are shown as pictures until you scroll back to them.") + .font(.caption).foregroundStyle(.secondary) + } + } + .formStyle(.grouped) + .frame(width: 480) + .padding(.vertical, 8) + } +} + +/// How many MCP tools the model has, with the list on click. +private struct ToolsBadge: View { + @ObservedObject var chat: ChatModel + @ObservedObject private var servers: MCPServerManagerObservable + @State private var showList = false + + init(chat: ChatModel) { + self.chat = chat + self.servers = chat.servers + } + + var body: some View { + let tools = chat.modelTools + let builtIn = chat.builtInToolNames + let count = tools.count + builtIn.count + Button { showList.toggle() } label: { + Label("\(count) tool\(count == 1 ? "" : "s")", systemImage: "wrench.and.screwdriver") + .labelStyle(.titleAndIcon) + } + .help("MCP tools the model can use") + .popover(isPresented: $showList) { + VStack(alignment: .leading, spacing: 6) { + Text("Tools the model can use").font(.headline) + ForEach(builtIn, id: \.self) { name in + HStack(spacing: 6) { + Text(name).font(.callout.monospaced()) + Text("built in").font(.caption2).foregroundStyle(.secondary) + } + } + if tools.isEmpty { + Text("No MCP tools. Turn tools on in the server list.").foregroundStyle(.secondary) + } + ForEach(tools, id: \.name) { tool in + HStack(spacing: 6) { + Text(tool.name).font(.callout.monospaced()) + if tool.app != nil { Image(systemName: "rectangle.on.rectangle").help("Shows an interactive view") } + } + } + Button("Server List…") { showList = false; chat.showServers = true }.padding(.top, 4) + } + .padding(14).frame(minWidth: 260, alignment: .leading) + } + } +} + diff --git a/examples/mcp-chat/Sources/MCPChat/MCPChatApp.swift b/examples/mcp-chat/Sources/MCPChat/MCPChatApp.swift new file mode 100644 index 0000000..9c933e4 --- /dev/null +++ b/examples/mcp-chat/Sources/MCPChat/MCPChatApp.swift @@ -0,0 +1,42 @@ +import AppKit +import LocalLMLabSDKCore +import SwiftUI + +// OAuth callbacks arrive as a custom-scheme URL. Route them through the AppDelegate, not SwiftUI's +// .onOpenURL — WindowGroup opens an extra window per open-URL event. +private final class AppDelegate: NSObject, NSApplicationDelegate { + func application(_ application: NSApplication, open urls: [URL]) { + for url in urls { MCPOAuthRedirectListener.shared.handleRedirect(url) } + } + + func applicationShouldTerminateAfterLastWindowClosed(_ sender: NSApplication) -> Bool { true } +} + +@main +struct MCPChatApp: App { + @NSApplicationDelegateAdaptor(AppDelegate.self) private var appDelegate + @StateObject private var chat: ChatModel + + init() { + // Distinct scheme (see Info.plist) so this app's OAuth callback can't reach another app. + MCPOAuthFlow.redirectURI = "mcpchat://oauth/callback" + _chat = StateObject(wrappedValue: ChatModel()) + } + + var body: some Scene { + WindowGroup { + ChatRootView(chat: chat) + } + .handlesExternalEvents(matching: []) + .defaultSize(width: 820, height: 760) + .commands { + CommandGroup(replacing: .newItem) { + Button("New Chat") { chat.newConversation() }.keyboardShortcut("n") + } + } + + Settings { + SettingsView() + } + } +} diff --git a/examples/mcp-chat/Sources/MCPChat/WidgetSlot.swift b/examples/mcp-chat/Sources/MCPChat/WidgetSlot.swift new file mode 100644 index 0000000..a764298 --- /dev/null +++ b/examples/mcp-chat/Sources/MCPChat/WidgetSlot.swift @@ -0,0 +1,211 @@ +import AppKit +import LocalLMLabSDKCore +import LocalLMLabSDKMCPAppsHost +import SwiftUI + +// A tool call's MCP App widget, inline under the call in the conversation. +// +// - First showing: the widget is fetched from the server, cached by hash and bound to the call. +// - Live widgets are capped (`MCPAppViewPool`, Settings → Live widgets). A widget released past the +// cap shows its snapshot until it scrolls back into view or is clicked, then it is re-created. +// - Re-creating (after the cap, after a relaunch, or after the session was rebuilt) checks the +// server first (`MCPAppRecreation.revalidate`): the current widget gets the stored result when the +// tool's output still has the same shape; otherwise the user is offered a refresh. Offline, the +// cached version is shown view-only. +// - Calls a widget makes go through the session (`MCPAppsSessionBackend`), so the session's +// authorizer asks the user exactly as it does for the model's calls, and they are recorded. + +struct WidgetSlot: View { + let record: ToolCallRecord + @ObservedObject var chat: ChatModel + @ObservedObject var pool: MCPAppViewPool + + enum Phase { + case idle + case loading + case live(MCPAppViewController, viewOnly: Bool, changed: Bool) + case needsRefresh(automatic: Bool) + case unavailable(String) + } + + @State private var phase: Phase = .idle + @State private var refreshing = false + + var body: some View { + VStack(alignment: .leading, spacing: 4) { + content + footer + } + .onAppear { if !pool.isLive(record.id) { Task { await activate() } } } + } + + @ViewBuilder private var content: some View { + switch phase { + case .live(let controller, _, _) where pool.isLive(record.id): + LiveWidget(controller: controller) + .onTapGesture { pool.touch(record.id) } + case .live, .idle: + if let snapshot = pool.snapshot(for: record.id) { + Image(nsImage: snapshot) + .resizable().aspectRatio(contentMode: .fit) + .frame(maxWidth: .infinity, maxHeight: 420) + .opacity(0.6) + .overlay { Label("Click to reactivate", systemImage: "play.circle").padding(8).background(.regularMaterial, in: Capsule()) } + .onTapGesture { Task { await activate() } } + } else { + placeholder("Interactive view", systemImage: "rectangle.on.rectangle") { + Button("Show") { Task { await activate() } } + } + } + case .loading: + placeholder("Loading view…", systemImage: nil) { ProgressView().controlSize(.small) } + case .needsRefresh(let automatic): + placeholder("This view changed since this answer. The saved result may not fit it.", + systemImage: "arrow.triangle.2.circlepath") { + Button(automatic ? "Refresh" : "Re-run tool…") { Task { await refresh() } } + .disabled(refreshing) + } + case .unavailable(let reason): + placeholder(reason, systemImage: "wifi.slash") { + Button("Retry") { Task { await activate() } } + } + } + } + + @ViewBuilder private var footer: some View { + let background = chat.session?.hostTranscript.appOnlyCallCount(for: record.id) ?? 0 + HStack(spacing: 10) { + if case .live(_, let viewOnly, let changed) = phase { + if viewOnly { Label("Offline — view only", systemImage: "wifi.slash") } + if changed { Label("Updated view", systemImage: "sparkles") } + } + if background > 0 { Label("\(background) background update\(background == 1 ? "" : "s")", systemImage: "arrow.clockwise") } + } + .font(.caption2).foregroundStyle(.secondary) + } + + private func placeholder(_ text: String, systemImage: String?, @ViewBuilder action: () -> some View) -> some View { + HStack(spacing: 10) { + if let systemImage { Image(systemName: systemImage).foregroundStyle(.secondary) } + Text(text).foregroundStyle(.secondary) + Spacer() + action() + } + .font(.callout) + .padding(12) + .background(.quaternary.opacity(0.4), in: RoundedRectangle(cornerRadius: 8)) + } + + // MARK: lifecycle + + private func activate() async { + if case .loading = phase { return } + guard let session = chat.session, let server = chat.serverID(of: record), let uri = record.app?.resourceURI, + record.mcpResult != nil else { return } + phase = .loading + let resource: MCPAppResource + var viewOnly = false + var changed = false + if chat.widgetCache.sha256(forRecord: record.id) == nil { + // First showing of this call: the server's current widget, kept and bound to the call. + do { + resource = try await chat.widgetCache.fetch(resourceURI: uri, server: server, manager: chat.manager, recordID: record.id) + } catch { + phase = .unavailable("Couldn't load the view (\(error.localizedDescription)).") + return + } + } else { + let check = await MCPAppRecreation.revalidate( + record, source: MCPServerManagerWidgetSource(manager: chat.manager, server: server), cache: chat.widgetCache) + switch check?.decision { + case .render(let offline)?: + guard let current = check?.resource else { phase = .unavailable("The view is no longer available."); return } + resource = current + viewOnly = offline + changed = check?.widgetChanged ?? false + case .needsRefresh(let automatic)?: + phase = .needsRefresh(automatic: automatic) + return + case .notApproved?: + phase = .unavailable("This version of the view isn't approved.") + return + case .unavailable?, nil: + phase = .unavailable("The server can't be reached and this view isn't saved.") + return + } + } + + let instance = record.id + let controller = pool.controller(for: instance) { + var actions = MCPAppsHostActions.session(session, instance: instance, openLink: { NSWorkspace.shared.open($0) }) + // Route widget messages through the chat so the turn shows as running and is saved. + actions.sendMessage = { [weak chat] text in chat?.sendFromWidget(text, instance: instance) } + return MCPAppViewController( + resource: resource, + tools: chat.manager.servers[server]?.tools ?? record.toolDescriptor.map { [$0] } ?? [], + backend: viewOnly ? OfflineBackend() as any MCPAppsBackend + : MCPAppsSessionBackend(session: session, manager: chat.manager, server: server, instance: instance), + configuration: MCPAppsBridgeConfiguration( + hostName: "MCP Chat", hostVersion: "0.1", + hostContext: ["displayMode": .string("inline"), "locale": .string(Locale.current.identifier(.bcp47))]), + bridgePolicy: ChatWidgetPolicy(viewOnly: viewOnly), + sandboxPolicy: .closed, + actions: actions) + } + MCPAppRecreation.deliver(record, to: controller) + phase = .live(controller, viewOnly: viewOnly, changed: changed) + await controller.load() + } + + /// Re-runs the call (as the host, through the session's authorizer). The fresh result arrives + /// as a new call at the end of the conversation, with the current view. + private func refresh() async { + guard let session = chat.session, let server = chat.serverID(of: record), case .mcp(_, let tool) = record.tool else { return } + refreshing = true + defer { refreshing = false } + let arguments = MCPAppRecreation.toolInput(from: record.arguments) + if case .failure(let error) = await session.callMCPTool(server: server, tool: tool, arguments: arguments, initiator: .host) { + chat.errorMessage = "Refresh failed: \(error)" + } + } +} + +/// Observes the controller so the slot follows the widget's reported height. +private struct LiveWidget: View { + @ObservedObject var controller: MCPAppViewController + + var body: some View { + MCPAppView(controller: controller) + .frame(height: max(80, controller.contentHeight ?? 360)) + .clipShape(RoundedRectangle(cornerRadius: 8)) + .overlay(RoundedRectangle(cornerRadius: 8).strokeBorder(.quaternary)) + } +} + +/// What a widget may ask the host for, on top of the protocol's own rules. Tool calls are allowed +/// here because the session's authorizer asks the user; messages and model context follow Settings. +struct ChatWidgetPolicy: MCPAppsBridgePolicy { + let viewOnly: Bool + + func authorizeToolCall(name: String, arguments: [String: MCPValue]) async -> MCPAppsDecision { + viewOnly ? .deny("Offline: this view is view-only") : .allow + } + + func authorizeOpenLink(_ url: URL) async -> MCPAppsDecision { + url.scheme?.lowercased() == "https" ? .allow : .deny("Only https links may be opened") + } + + func authorizeMessage(text: String) async -> MCPAppsDecision { + ChatSettings.allowWidgetMessages ? .allow : .deny("Widget messages are off in Settings") + } + + func authorizeModelContextUpdate() async -> MCPAppsDecision { + ChatSettings.allowWidgetContext ? .allow : .deny("Widget context is off in Settings") + } +} + +/// The backend for a view-only (offline) widget: nothing reaches the server. +struct OfflineBackend: MCPAppsBackend { + func callTool(name: String, arguments: [String: MCPValue]) async -> Result { .failure(.notConnected) } + func readResource(uri: String) async -> Result { .failure(.notConnected) } +} diff --git a/examples/mcp-chat/packaging/Info.plist b/examples/mcp-chat/packaging/Info.plist new file mode 100644 index 0000000..def8478 --- /dev/null +++ b/examples/mcp-chat/packaging/Info.plist @@ -0,0 +1,25 @@ + + + + + CFBundleDisplayNameMCP Chat + CFBundleExecutableMCPChat + CFBundleIdentifierlab.locallm.mcpapps.chat + CFBundleNameMCP Chat + CFBundlePackageTypeAPPL + CFBundleShortVersionString0.1.0 + CFBundleVersion0.1.0 + LSApplicationCategoryTypepublic.app-category.developer-tools + LSMinimumSystemVersion27.0 + NSHighResolutionCapable + + CFBundleURLTypes + + + CFBundleURLNamelab.locallm.mcpapps.chat.oauth + CFBundleURLSchemes + mcpchat + + + + diff --git a/examples/mcp-chat/packaging/MCPChat.entitlements b/examples/mcp-chat/packaging/MCPChat.entitlements new file mode 100644 index 0000000..5b7ef2e --- /dev/null +++ b/examples/mcp-chat/packaging/MCPChat.entitlements @@ -0,0 +1,10 @@ + + + + + + com.apple.security.app-sandbox + com.apple.security.network.client + + diff --git a/examples/mcp-chat/packaging/build-and-sign.sh b/examples/mcp-chat/packaging/build-and-sign.sh new file mode 100755 index 0000000..e6c89d0 --- /dev/null +++ b/examples/mcp-chat/packaging/build-and-sign.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Builds and signs "MCP Chat.app" (unnotarized, local use). Works with either way of getting the SDK: +# - binary (the published SDK release): Core and Inference are .frameworks next to the binary, +# each carrying its own resource bundles (the MLX Metal shaders); +# - source (path dependencies): Core and Inference are dynamic libraries next to the binary, and +# the MLX runtime's resource bundles go in Contents/Resources, where SwiftPM's lookup finds them +# (building the SDK from source needs the Metal Toolchain). +# Needs Xcode 27 (Swift 6.4). +# +# APP_IDENTITY codesigning identity (default: the first "Apple Development" identity; "-" = ad hoc) +# CONFIG debug | release (default: release) +# SANDBOX 1 (default) | 0 — 0 signs without App Sandbox, for development only: the app can +# then use an existing model cache (MCPCHAT_MODEL_CACHE) instead of downloading + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +APP_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" +APP_NAME="MCP Chat" +BIN_NAME="MCPChat" +CONFIG="${CONFIG:-release}" +DEVELOPER_DIR="${DEVELOPER_DIR:-$(xcode-select -p)}" +export DEVELOPER_DIR +APP_IDENTITY="${APP_IDENTITY:-$(security find-identity -v -p codesigning | grep 'Apple Development' | head -1 | sed -E 's/.*"(.*)".*/\1/' || true)}" +APP_IDENTITY="${APP_IDENTITY:--}" + +BUILD_DIR="$APP_ROOT/build" +APP_DIR="$APP_ROOT/dist/${APP_NAME}.app" +rm -rf "$APP_DIR" +mkdir -p "$BUILD_DIR" "$APP_DIR/Contents/MacOS" "$APP_DIR/Contents/Resources" + +swift build --package-path "$APP_ROOT" -c "$CONFIG" --arch arm64 --build-path "$BUILD_DIR/swift" +BIN_DIR="$(swift build --package-path "$APP_ROOT" -c "$CONFIG" --arch arm64 --build-path "$BUILD_DIR/swift" --show-bin-path)" + +cp "$SCRIPT_DIR/Info.plist" "$APP_DIR/Contents/Info.plist" +cp "$BIN_DIR/$BIN_NAME" "$APP_DIR/Contents/MacOS/$BIN_NAME" +shopt -s nullglob +for lib in "$BIN_DIR"/*.dylib; do cp "$lib" "$APP_DIR/Contents/MacOS/"; done +for bundle in "$BIN_DIR"/*.bundle; do cp -R "$bundle" "$APP_DIR/Contents/Resources/"; done +FRAMEWORKS=() +for fw in "$BIN_DIR"/LocalLMLabSDK*.framework; do + name="$(basename "$fw")" + cp -R "$fw" "$APP_DIR/Contents/MacOS/$name" + # The published xcframework zips carry AppleDouble/xattr detritus that breaks signing. + find "$APP_DIR/Contents/MacOS/$name" -name '._*' -delete + xattr -cr "$APP_DIR/Contents/MacOS/$name" + FRAMEWORKS+=("$APP_DIR/Contents/MacOS/$name") +done +printf 'APPL????' > "$APP_DIR/Contents/PkgInfo" + +echo "Signing as: $APP_IDENTITY" +ENT="$SCRIPT_DIR/MCPChat.entitlements" +if [[ "${SANDBOX:-1}" == "0" ]]; then + ENT="$BUILD_DIR/MCPChat-dev.entitlements" + printf '\n\n\n' > "$ENT" + echo "Development build: App Sandbox OFF" +fi +for lib in "$APP_DIR"/Contents/MacOS/*.dylib; do codesign --force --sign "$APP_IDENTITY" "$lib"; done +# Frameworks inside-out: nested bundles, the framework binary, then the framework. +for fw in ${FRAMEWORKS[@]+"${FRAMEWORKS[@]}"}; do # (empty-array safe under set -u in bash 3.2) + while IFS= read -r b; do codesign --force --sign "$APP_IDENTITY" "$b"; done < <(find "$fw" -type d -name '*.bundle') + codesign --force --sign "$APP_IDENTITY" "$fw/$(basename "$fw" .framework)" + codesign --force --sign "$APP_IDENTITY" "$fw" +done +codesign --force --entitlements "$ENT" --sign "$APP_IDENTITY" "$APP_DIR/Contents/MacOS/$BIN_NAME" +codesign --force --entitlements "$ENT" --sign "$APP_IDENTITY" "$APP_DIR" +codesign --verify --deep --strict --verbose=2 "$APP_DIR" +echo "$APP_DIR" diff --git a/examples/security-demo/README.md b/examples/security-demo/README.md index b11e971..9d9b740 100644 --- a/examples/security-demo/README.md +++ b/examples/security-demo/README.md @@ -88,7 +88,7 @@ gated. For tools that *are* in the session, "Confirm each" decides whether a call runs or asks first. The approval is set where the tools come from — per MCP server on `lab.mcp`, and for the app's -own tools on the authorizer (docs/sdk-authority-model.md §9): +own tools on the authorizer ([SDK guide §7c](../../docs/sdk-guide.md#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first)): ```swift // Todoist (an MCP server): nil = the untrusted default, ask before each call; .allow = don't ask. From 6b0a1a46809b053bd0f53a08831ca06f362d2716 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:14:20 -0700 Subject: [PATCH 16/73] docs: annotate examples/mcp-chat MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds an examples/mcp-chat section to annotated-examples.md: ChatModel.swift, WidgetSlot.swift and MCPChatApp.swift in full, the SDK-facing views of ChatViews.swift (plain SwiftUI elided), every SDK line marked. New marker `// ← SDK (MCPAppsHost)` for the MCP Apps package; legend, size table and intro updated. Tally: 139 marked lines (86 SDK, 39 MCPAppsHost, 4 Inference, 10 Components). Co-Authored-By: Claude --- docs/annotated-examples.md | 1065 +++++++++++++++++++++++++++++++++++- 1 file changed, 1061 insertions(+), 4 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index 68ded01..8954deb 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -2,8 +2,9 @@ The full source of every reference app, with every line that actually touches the SDK marked `// ← SDK` (Core), `// ← SDK (Inference)` (the MLX runtime — `code-buddy`, `repo-qa-local`, -`workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, and `mlx-control-room`), `// ← SDK (Remote)` (online providers — -`model-switch` and `security-demo`), or `// ← Components` (`components-demo` and `components-updates-demo`). Everything else is ordinary SwiftUI/Foundation — the point +`workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, `mlx-control-room`, and `mcp-chat`), `// ← SDK (Remote)` (online providers — +`model-switch` and `security-demo`), `// ← SDK (MCPAppsHost)` (showing MCP servers' interactive views — +`mcp-chat`), or `// ← Components` (`components-demo`, `components-updates-demo`, and `mcp-chat`). Everything else is ordinary SwiftUI/Foundation — the point of marking it this way is to make obvious just how little of each file is SDK-specific plumbing. `plate-today` and `plate-today-tools` are a matched pair — the same app twice, "Path B" (hand- written `Tool` adapters) vs. "Path A" (Core's ready-made ones, `// ← SDK (Path A)`) — meant to be @@ -39,11 +40,13 @@ non-comment lines; these examples are commented far more heavily than production | [`code-buddy`](#examplescode-buddy) | 315 | 413 | a CLI coding agent: two models with routing, workspace + host `Process` tools, MCP, a persistent REPL session (2 files) | | [`aiql`](#examplesaiql) | 395 | 494 | a plain-English request → one read-only SQL `SELECT` over an MCP dataset → the CSV you asked for, sandboxed SwiftUI, zero fabricated values; a pinned default model and a trust policy for the free-text picker | | [`vistanova`](#examplesvistanova) | 931 | 1,294 | a tiny local search engine: web search through a Tavily MCP server on one local model, summaries from a **pinned** MLX model on another; defends against a model that skips the tool call (7 files) | +| [`mcp-chat`](#examplesmcp-chat) | 938 | 1,026 | a chat with a local model where an MCP server's own interactive view (Todoist's task list) shows up under the tool call: the conversation is `hostTranscript`, the view is `MCPAppsHost` (4 files, views excerpted) | | [`components-updates-demo`](#examplescomponents-updates-demo) | 151 | 170 | the `Components` model **onboarding**, **update** and **versions** views, driven by simulated sources so every state is reachable | | [`mlx-control-room`](#examplesmlx-control-room) | 1,434 | 1,812 | every MLX knob with a gauge, plus the supply-chain flow made visible: validate, download, **pin**, update, roll back, clean up (excerpted; UI omitted) | -The SDK-specific part of each — the lines carrying a `// ← SDK` marker — is a few dozen at most, -and each section's **Tally** breaks that down. The rest is ordinary SwiftUI, Foundation, and +The SDK-specific part of each — the lines carrying a `// ← SDK` marker — is a few dozen at most +(`mcp-chat`, whose conversation UI *is* the SDK's chat history, is the exception), and each +section's **Tally** breaks that down. The rest is ordinary SwiftUI, Foundation, and argument parsing. ## `examples/plate-today` @@ -3632,3 +3635,1057 @@ which puts a ready-made `Components` UI on the same flows, and [`vistanova`](#ex minimal end (one `pinnedRevisions:` and nothing else). The one thing worth copying verbatim is `AllowMLXCommunityOrShipped`: an allow-list keyed on `mlx-community/` plus the exact artifacts the app itself ships, because a curated adapter lives in a different namespace. + +## `examples/mcp-chat` + +*938 lines of code across 4 files (1,026 with comments). `ChatModel.swift`, `WidgetSlot.swift` +and `MCPChatApp.swift` are below in full; from `ChatViews.swift` the views that read the SDK are +shown and the plain-SwiftUI ones (bubbles, composer, settings, the first-run screen) are elided, +marked `// …`.* + +A chat with a local model (Qwen3 8B through MLX, or Apple's on-device model) where a tool call +that has an **MCP App** — a server's own interactive view, such as Todoist's task list — shows +that view in the conversation ([`sdk-guide.md` §3f](sdk-guide.md#3f-mcp-apps-showing-a-servers-interactive-views)). +The model never asks for a view: it calls a tool, and if the tool's listing names a view, the app +draws it under the call. Three SDK pieces carry the app: + +- **`session.hostTranscript`** is the whole conversation UI — user bubbles, tool-call rows (live + records with the full result), the streaming reply, reasoning split from the answer + ([Building a chat app](sdk-guide.md#building-a-chat-app--hosttranscript-streamresponse-turncontext)). + The view never touches Apple's transcript. +- **`LocalLMLabSDKMCPAppsHost`** renders a view in a sandboxed web view, routes the view's own + tool calls back through the session (so they meet the same approval as the model's), keeps a + few views live and the rest as snapshots, and caches each view by hash so it comes back after a + relaunch. Everything marked `// ← SDK (MCPAppsHost)` is that package — open source, built on + Core's public API. +- **`lab.mcp` + `Components`**: the one MCP manager, set up to declare MCP Apps support; the + server list (with each server's trust and approval) and the approval sheet are prebuilt views. + +Links `LocalLMLabSDKCore`, `LocalLMLabSDKInference`, `LocalLMLabSDKComponents` and +`LocalLMLabSDKMCPAppsHost`. App Sandbox with outbound network only. + +### `Sources/MCPChat/ChatModel.swift` + +The app's state: the lab, the session, the server list, the widget cache and pool. Note what +isn't here: no tool list is rebuilt when the user turns a tool on (a session follows `lab.mcp` on +its own), no approval policy (each server's trust and approval live on `lab.mcp`, set from the +server list), no `` parsing (`hostTranscript` splits it). A new session is made only when +the user switches models — carrying both the model's memory and the transcript over. + +```swift +import AppKit +import Combine +import Foundation +import FoundationModels +import LocalLMLabSDKComponents // ← Components +import LocalLMLabSDKCore // ← SDK +import LocalLMLabSDKInference // ← SDK (Inference) +import LocalLMLabSDKMCPAppsHost // ← SDK (MCPAppsHost) + +// The model this app routes to by default, pinned to the commit it was tried against (see +// workspace-buddy-local for why a revision is pinned). +let qwenRepo = "mlx-community/Qwen3-8B-4bit" +let qwenRevision = "545dc4251c05440727734bcd94334791f6ab0192" + +extension RouteName { // ← SDK + static let chat: RouteName = "chat" // ← SDK +} + +enum ChatModelChoice: String, CaseIterable, Identifiable { + case qwen3 + case apple + + var id: String { rawValue } + var title: String { + switch self { + case .qwen3: "Qwen3 8B (MLX, local)" + case .apple: "Apple on-device model" + } + } + var modelID: ModelID { + switch self { + case .qwen3: ModelID(scheme: "mlx", rest: qwenRepo)! // ← SDK + case .apple: .system + } + } +} + +/// App settings (Settings window). Read where they are used, so a change applies from the next +/// turn without restarting. +enum ChatSettings { + static let modelKey = "model" + /// Whether a reopened conversation continues with the model's memory of it. Off: the user still + /// sees the earlier conversation, but the model starts fresh (and nothing the model saw is + /// written to disk). + static let rememberKey = "rememberConversation" + static let widgetMessagesKey = "allowWidgetMessages" + static let widgetContextKey = "allowWidgetContext" + static let liveWidgetsKey = "liveWidgetLimit" + static let clockToolKey = "clockTool" + + static func register() { + UserDefaults.standard.register(defaults: [ + modelKey: ChatModelChoice.qwen3.rawValue, + rememberKey: true, + widgetMessagesKey: false, + widgetContextKey: true, + liveWidgetsKey: 3, + clockToolKey: true, + ]) + } + + static var model: ChatModelChoice { + ChatModelChoice(rawValue: UserDefaults.standard.string(forKey: modelKey) ?? "") ?? .qwen3 + } + static var remember: Bool { UserDefaults.standard.bool(forKey: rememberKey) } + static var allowWidgetMessages: Bool { UserDefaults.standard.bool(forKey: widgetMessagesKey) } + static var allowWidgetContext: Bool { UserDefaults.standard.bool(forKey: widgetContextKey) } + static var liveWidgets: Int { max(1, UserDefaults.standard.integer(forKey: liveWidgetsKey)) } + static var clockTool: Bool { UserDefaults.standard.bool(forKey: clockToolKey) } +} + +@MainActor +final class ChatModel: ObservableObject { + enum Status: Equatable { + /// The chosen model is not ready (e.g. Qwen3 is not downloaded yet). + case needsModel + case downloading(Double) + case ready + case responding + } + + let manager: MCPServerManager + let servers: MCPServerManagerObservable // ← Components + let toolConfirm = ToolConfirmationPresenter() // ← Components + let widgetCache = MCPAppWidgetCache() // ← SDK (MCPAppsHost) + let pool = MCPAppViewPool(limit: ChatSettings.liveWidgets) // ← SDK (MCPAppsHost) + + @Published private(set) var session: LocalLMLabSession? + @Published private(set) var status: Status = .needsModel + @Published private(set) var modelChoice = ChatSettings.model + /// Set when a conversation was reopened without the model's memory: turns up to this one were + /// not given to the model. + @Published private(set) var memoryBoundaryTurn: Int? + @Published var errorMessage: String? + @Published var showServers = false + /// One authorizer for every session. It follows each server's trust and tool approval, which + /// the user sets per server in the server list (Components' picker), and remembers a widget's + /// answers; a session made after a model switch keeps them. + private let authorizer: ConfirmingToolAuthorizer // ← SDK + + // Development only: MCPCHAT_MODEL_CACHE points at an existing Hugging Face cache, so a build + // without App Sandbox (SANDBOX=0 packaging/build-and-sign.sh) can skip the download. A + // sandboxed build can't read outside its container and downloads into it. + private let mlx = MLXModelProvider( // ← SDK (Inference) + cacheDirectory: ProcessInfo.processInfo.environment["MCPCHAT_MODEL_CACHE"].map { URL(fileURLWithPath: $0) }, + residentModelLimit: 1, pinnedRevisions: [qwenRepo: qwenRevision]) + private let lab: LocalLMLab + private let store = ConversationStore() + private var sessionSignature: String? + private var cancellables: Set = [] + + static let instructions = """ + You are a helpful assistant in a chat app. Use the tools you have when the user asks about \ + their data or asks you to change it. Some tools also show the user an interactive view of \ + the result; when one does, keep your reply short and do not repeat what the view shows. + """ + + init() { + ChatSettings.register() + // The lab makes the one MCP manager (`lab.mcp`). Declare MCP Apps support so servers that + // gate their widgets on it offer them. + let lab = LocalLMLab(configuration: .init( // ← SDK + providers: [mlx, SystemModelProvider()], // ← SDK + mcp: MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps()))) // ← SDK — MCPSettings; ← SDK (MCPAppsHost) — .advertisingMCPApps() + self.lab = lab + self.manager = lab.mcp // ← SDK + self.servers = MCPServerManagerObservable(core: lab.mcp) // ← Components + authorizer = ConfirmingToolAuthorizer(channel: toolConfirm) // ← SDK + modelChoice = ChatSettings.model + lab.models.route(.chat, to: modelChoice.modelID) // ← SDK + } + + // MARK: launch + + func start() async { + await restoreServers() + refreshModelStatus() + if status == .ready { openSavedConversation() } + + NotificationCenter.default.publisher(for: UserDefaults.didChangeNotification) + .receive(on: RunLoop.main) + .sink { [weak self] _ in self?.settingsChanged() } + .store(in: &cancellables) + NotificationCenter.default.publisher(for: NSApplication.willTerminateNotification) + .sink { [weak self] _ in MainActor.assumeIsolated { self?.save() } } + .store(in: &cancellables) + } + + private func refreshModelStatus() { + switch lab.models.availability(for: modelChoice.modelID) { // ← SDK + case .available: if status != .responding { status = .ready } + default: status = .needsModel + } + } + + var modelUnavailableReason: String? { + switch lab.models.availability(for: modelChoice.modelID) { // ← SDK + case .available, .notDownloaded: nil + case .needsCredential: "This model needs a credential." + case .unavailable(_, let detail): detail + @unknown default: "This model is not available." + } + } + + func downloadModel() async { + guard modelChoice == .qwen3, status == .needsModel else { return } + status = .downloading(0) + do { + if let preflight = try? await mlx.validate(qwenRepo), !preflight.passed { // ← SDK (Inference) + throw ChatError("This Mac can't run \(qwenRepo): \(preflight.detail ?? "pre-flight failed").") + } + for try await event in mlx.download(qwenRepo) { // ← SDK (Inference) + if case .progress(_, _, let fraction) = event { status = .downloading(fraction) } + } + } catch { + errorMessage = "Download failed: \(error.localizedDescription)" + } + status = .needsModel + refreshModelStatus() + if status == .ready, session == nil { openSavedConversation() } + } + + private func settingsChanged() { + pool.limit = ChatSettings.liveWidgets // ← SDK (MCPAppsHost) + if !ChatSettings.remember { store.forgetModelMemory() } + guard ChatSettings.model != modelChoice, status != .responding else { return } + modelChoice = ChatSettings.model + lab.models.route(.chat, to: modelChoice.modelID) // ← SDK + refreshModelStatus() + // The session is rebuilt on the next turn, carrying the conversation over to the new model. + if status == .ready, session == nil { openSavedConversation() } + } + + // MARK: conversation + + /// Reopens the saved conversation: always what the user saw; the model's memory of it only + /// when "Remember conversations" is on. + private func openSavedConversation() { + let saved = store.load() + let memory = ChatSettings.remember ? saved?.model : nil + do { + let session = try makeSession(restoring: memory) + if let shown = saved?.shown { try session.hostTranscript.restore(from: shown) } // ← SDK + if memory == nil, let last = session.hostTranscript.entries.last?.turn { memoryBoundaryTurn = last } // ← SDK + install(session) + } catch { + errorMessage = "Couldn't open the saved conversation: \(error.localizedDescription)" + if let fresh = try? makeSession(restoring: nil) { install(fresh) } + } + } + + func newConversation() { + guard status != .responding else { return } + Task { await pool.releaseAll() } // ← SDK (MCPAppsHost) + store.clear() + memoryBoundaryTurn = nil + session = nil + sessionSignature = nil + if status == .ready, let fresh = try? makeSession(restoring: nil) { install(fresh) } + } + + func send(_ text: String) { + let prompt = text.trimmingCharacters(in: .whitespacesAndNewlines) + guard !prompt.isEmpty else { return } + Task { await respond { $0.streamResponse(to: prompt) } } // ← SDK + } + + /// A widget's `ui/message` — only reached when Settings allows widget messages. + func sendFromWidget(_ text: String, instance: String) { + Task { await respond { $0.streamResponse(to: text, fromAppInstance: instance) } } // ← SDK + } + + /// Runs a streamed turn. The view draws the reply as it arrives from + /// `hostTranscript.replyInProgress`, so the stream only needs draining here. + private func respond(_ turn: (LocalLMLabSession) -> AsyncThrowingStream) async { + guard status == .ready else { return } + let session: LocalLMLabSession + do { + session = try currentSession() + } catch { + errorMessage = error.localizedDescription + return + } + status = .responding + defer { + status = .ready + save() + } + do { + for try await _ in turn(session) {} + } catch { + errorMessage = await GenerationErrorDescription.describe(error) // ← SDK + } + } + + /// The session for the next turn. A session follows `lab.mcp` by itself (tools turned on or + /// off, servers added, trust changed: picked up on its next turn). What it can't follow is a + /// different model or a change to the app's own tools; then a new session takes over the + /// conversation: the model's memory (its transcript) and what the user sees (the archive). + private func currentSession() throws -> LocalLMLabSession { + if let session, sessionSignature == signature() { return session } + guard let old = session else { + let fresh = try makeSession(restoring: nil) + install(fresh) + return fresh + } + let fresh = try makeSession(restoring: old.languageModelSession.transcript) // ← SDK + try fresh.hostTranscript.restore(from: old.hostTranscript.archive()) // ← SDK + // Live widgets call through the session they were made with: re-create them on the new one. + Task { await pool.releaseAll() } // ← SDK (MCPAppsHost) + install(fresh) + return fresh + } + + private func makeSession(restoring memory: Transcript?) throws -> LocalLMLabSession { + // A saved conversation carries its own instructions; a new one gets the app's. + var session = try lab.makeSession( // ← SDK + route: .chat, tools: builtInTools, instructions: memory == nil ? Self.instructions : nil, + restoring: memory, mcpAppHints: true, authorizer: authorizer) + session.retryOnContextOverflow = RetryPolicy(maxRetries: 2, compact: Self.dropOldestHalf) // ← SDK + session.turnContext = { Self.clock(Date()) } // ← SDK + sessionSignature = signature() + return session + } + + /// The app's own tools, alongside the MCP servers'. The SDK's clock (`getCurrentTime`, rated + /// read-only, so it never asks) lets the model check the date when it needs to — e.g. to turn + /// "tomorrow" into a date for Todoist. It is in addition to the date sent with each turn. + private var builtInTools: [any Tool] { + ChatSettings.clockTool ? [ClockTool()] : [] // ← SDK + } + + var builtInToolNames: [String] { builtInTools.map(\.name) } + + private func install(_ session: LocalLMLabSession) { + self.session = session + } + + /// What a session can't change by itself: the model and the app's own tools. + private func signature() -> String { + ([modelChoice.rawValue] + builtInToolNames).joined(separator: "\n") + } + + /// The current date and time, sent with every turn: a model has no clock, so without it + /// "what's due tomorrow?" resolves against a date from its training data. + nonisolated static func clock(_ now: Date, timeZone: TimeZone = .current) -> String { + var calendar = Calendar(identifier: .gregorian) + calendar.timeZone = timeZone + let parts = calendar.dateComponents([.year, .month, .day, .hour, .minute, .weekday], from: now) + let weekday = calendar.weekdaySymbols[(parts.weekday ?? 1) - 1] + let offset = timeZone.secondsFromGMT(for: now) / 60 + let date = String(format: "%04d-%02d-%02d", parts.year ?? 0, parts.month ?? 0, parts.day ?? 0) + let time = String(format: "%02d:%02d", parts.hour ?? 0, parts.minute ?? 0) + let utc = String(format: "UTC%@%02d:%02d", offset < 0 ? "-" : "+", abs(offset) / 60, abs(offset) % 60) + return "[Current date and time: \(weekday) \(date) \(time), \(timeZone.identifier) (\(utc)). " + + "Use it for relative dates such as today, tomorrow or next week.]" + } + + /// Context overflow (e.g. a long conversation moved to the smaller on-device model): keep the + /// instructions and the newer half of the conversation, starting at a user turn. + nonisolated static func dropOldestHalf(_ transcript: Transcript) -> Transcript { + let entries = Array(transcript) + let prompts = entries.indices.filter { if case .prompt = entries[$0] { true } else { false } } + guard prompts.count > 1 else { return transcript } + let keepFrom = prompts[prompts.count / 2] + var kept: [Transcript.Entry] = [] + if let first = entries.first, case .instructions = first { kept.append(first) } + kept.append(contentsOf: entries[keepFrom...]) + return Transcript(entries: kept) + } + + private func save() { + guard let session else { return } + store.save(shown: try? session.hostTranscript.archive(), // ← SDK + model: ChatSettings.remember ? session.languageModelSession.transcript : nil) // ← SDK + } + + // MARK: servers (saved list; quiet reconnect) + + private struct SavedServer: Codable { + var url: URL + var name: String + var authType: MCPAuthType // ← SDK + var manualClientID: String? + var noAuth: Bool + /// The server's tools as last seen, with the user's on/off choice for each. New tools from + /// a server start off (the SDK's default); restoring these keeps the user's choices. + var tools: [MCPToolDescriptor]? // ← SDK + /// The user's trust and tool approval for the server (server list). The manager doesn't + /// persist them; they are set again after restore. + var trust: MCPServerTrust? // ← SDK + var toolApproval: ToolApproval? // ← SDK + } + + private let savedServersKey = "servers.v1" + + private func restoreServers() async { + let saved = (UserDefaults.standard.data(forKey: savedServersKey) + .flatMap { try? JSONDecoder().decode([SavedServer].self, from: $0) }) ?? [] + manager.restore(from: saved.map { // ← SDK + (id: MCPServerID(rawValue: $0.url.absoluteString), url: $0.url, displayName: $0.name, tools: $0.tools ?? [], estimatedTokens: 0, // ← SDK + enabled: true, authType: $0.authType, manualClientID: $0.manualClientID, resources: []) + }) + // Trust before reconnecting: trusting records the saved tool list, so a server whose tools + // changed while the app was closed shows "tools changed since trusted". + for server in saved { + let id = MCPServerID(rawValue: server.url.absoluteString) // ← SDK + if let trust = server.trust { manager.setTrust(trust, server: id) } // ← SDK + manager.setToolApproval(server.toolApproval, server: id) // ← SDK + } + // Only servers that need no sign-in, or have a stored credential, reconnect at launch: this + // app never opens a browser unprompted. + for server in saved where server.noAuth || MCPCredentialProbe.storedKind(for: server.url) != nil { // ← SDK + _ = await manager.reconnect(MCPServerID(rawValue: server.url.absoluteString)) // ← SDK + } + servers.$servers.dropFirst().sink { [weak self] in self?.persist($0) }.store(in: &cancellables) // ← Components + } + + private func persist(_ states: [MCPServerID: MCPServerState]) { // ← SDK + let previous = (UserDefaults.standard.data(forKey: savedServersKey) + .flatMap { try? JSONDecoder().decode([SavedServer].self, from: $0) }) ?? [] + let saved = states.values.map { state in + SavedServer(url: state.url, name: state.displayName, authType: state.authType, manualClientID: state.manualClientID, + noAuth: state.connectionStatus == .connected + ? MCPCredentialProbe.storedKind(for: state.url) == nil // ← SDK + : previous.first { $0.url == state.url }?.noAuth ?? false, + tools: state.tools.isEmpty ? previous.first { $0.url == state.url }?.tools : state.tools, + trust: state.trust, toolApproval: state.toolApproval) + } + if let data = try? JSONEncoder().encode(saved) { UserDefaults.standard.set(data, forKey: savedServersKey) } + } + + /// The MCP tools the model is given on its next turn. + var modelTools: [MCPToolDescriptor] { manager.toolsForSession() } // ← SDK + + /// Connected servers none of whose tools are on: the model can't use them yet. (New tools start + /// off; the user turns on the ones they want in the server list.) + var serversWithNoToolsOn: [MCPServerState] { // ← SDK + servers.sortedServers.filter { state in // ← Components + state.connectionStatus == .connected && !state.tools.isEmpty + && !state.tools.contains { $0.enabled && $0.isModelVisible } // ← SDK + } + } + + /// The server a tool call went to, for its widget. + func serverID(of record: ToolCallRecord) -> MCPServerID? { // ← SDK + if case .mcp(let server, _) = record.tool { return server } // ← SDK + return nil + } +} + +struct ChatError: LocalizedError { + let errorDescription: String? + init(_ message: String) { errorDescription = message } +} + +/// The open conversation on disk (Application Support/MCPChat): what the user saw +/// (`HostTranscript.archive()`) and, when "Remember conversations" is on, the model's transcript. +struct ConversationStore { + struct Saved { + var shown: Data? + var model: Transcript? + } + + private let directory: URL = { + let base = FileManager.default.urls(for: .applicationSupportDirectory, in: .userDomainMask)[0] + .appendingPathComponent("MCPChat/Conversation", isDirectory: true) + try? FileManager.default.createDirectory(at: base, withIntermediateDirectories: true) + return base + }() + + private var shownFile: URL { directory.appendingPathComponent("shown.json") } + private var modelFile: URL { directory.appendingPathComponent("model.json") } + + func load() -> Saved? { + let shown = try? Data(contentsOf: shownFile) + let model = (try? Data(contentsOf: modelFile)).flatMap { try? JSONDecoder().decode(Transcript.self, from: $0) } + return shown == nil && model == nil ? nil : Saved(shown: shown, model: model) + } + + func save(shown: Data?, model: Transcript?) { + if let shown { try? shown.write(to: shownFile, options: .atomic) } + if let model, let data = try? JSONEncoder().encode(model) { + try? data.write(to: modelFile, options: .atomic) + } else { + forgetModelMemory() + } + } + + func forgetModelMemory() { + try? FileManager.default.removeItem(at: modelFile) + } + + func clear() { + try? FileManager.default.removeItem(at: shownFile) + forgetModelMemory() + } +} +``` + +### `Sources/MCPChat/WidgetSlot.swift` + +One tool call's view. First showing: fetch the server's view, cache it by hash, bind it to the +call. Later (scrolled back past the live limit, after a relaunch, after a session switch): +`MCPAppRecreation.revalidate` decides whether the stored result still fits the server's current +view. The view's own calls go through `MCPAppsSessionBackend` → the session's authorizer. + +```swift +import AppKit +import LocalLMLabSDKCore // ← SDK +import LocalLMLabSDKMCPAppsHost // ← SDK (MCPAppsHost) +import SwiftUI + +// A tool call's MCP App widget, inline under the call in the conversation. +// +// - First showing: the widget is fetched from the server, cached by hash and bound to the call. +// - Live widgets are capped (`MCPAppViewPool`, Settings → Live widgets). A widget released past the +// cap shows its snapshot until it scrolls back into view or is clicked, then it is re-created. +// - Re-creating (after the cap, after a relaunch, or after the session was rebuilt) checks the +// server first (`MCPAppRecreation.revalidate`): the current widget gets the stored result when the +// tool's output still has the same shape; otherwise the user is offered a refresh. Offline, the +// cached version is shown view-only. +// - Calls a widget makes go through the session (`MCPAppsSessionBackend`), so the session's +// authorizer asks the user exactly as it does for the model's calls, and they are recorded. + +struct WidgetSlot: View { + let record: ToolCallRecord // ← SDK + @ObservedObject var chat: ChatModel + @ObservedObject var pool: MCPAppViewPool // ← SDK (MCPAppsHost) + + enum Phase { + case idle + case loading + case live(MCPAppViewController, viewOnly: Bool, changed: Bool) // ← SDK (MCPAppsHost) + case needsRefresh(automatic: Bool) + case unavailable(String) + } + + @State private var phase: Phase = .idle + @State private var refreshing = false + + var body: some View { + VStack(alignment: .leading, spacing: 4) { + content + footer + } + .onAppear { if !pool.isLive(record.id) { Task { await activate() } } } // ← SDK (MCPAppsHost) + } + + @ViewBuilder private var content: some View { + switch phase { + case .live(let controller, _, _) where pool.isLive(record.id): // ← SDK (MCPAppsHost) + LiveWidget(controller: controller) + .onTapGesture { pool.touch(record.id) } // ← SDK (MCPAppsHost) + case .live, .idle: + if let snapshot = pool.snapshot(for: record.id) { // ← SDK (MCPAppsHost) + Image(nsImage: snapshot) + .resizable().aspectRatio(contentMode: .fit) + .frame(maxWidth: .infinity, maxHeight: 420) + .opacity(0.6) + .overlay { Label("Click to reactivate", systemImage: "play.circle").padding(8).background(.regularMaterial, in: Capsule()) } + .onTapGesture { Task { await activate() } } + } else { + placeholder("Interactive view", systemImage: "rectangle.on.rectangle") { + Button("Show") { Task { await activate() } } + } + } + case .loading: + placeholder("Loading view…", systemImage: nil) { ProgressView().controlSize(.small) } + case .needsRefresh(let automatic): + placeholder("This view changed since this answer. The saved result may not fit it.", + systemImage: "arrow.triangle.2.circlepath") { + Button(automatic ? "Refresh" : "Re-run tool…") { Task { await refresh() } } + .disabled(refreshing) + } + case .unavailable(let reason): + placeholder(reason, systemImage: "wifi.slash") { + Button("Retry") { Task { await activate() } } + } + } + } + + @ViewBuilder private var footer: some View { + let background = chat.session?.hostTranscript.appOnlyCallCount(for: record.id) ?? 0 // ← SDK + HStack(spacing: 10) { + if case .live(_, let viewOnly, let changed) = phase { + if viewOnly { Label("Offline — view only", systemImage: "wifi.slash") } + if changed { Label("Updated view", systemImage: "sparkles") } + } + if background > 0 { Label("\(background) background update\(background == 1 ? "" : "s")", systemImage: "arrow.clockwise") } + } + .font(.caption2).foregroundStyle(.secondary) + } + + private func placeholder(_ text: String, systemImage: String?, @ViewBuilder action: () -> some View) -> some View { + HStack(spacing: 10) { + if let systemImage { Image(systemName: systemImage).foregroundStyle(.secondary) } + Text(text).foregroundStyle(.secondary) + Spacer() + action() + } + .font(.callout) + .padding(12) + .background(.quaternary.opacity(0.4), in: RoundedRectangle(cornerRadius: 8)) + } + + // MARK: lifecycle + + private func activate() async { + if case .loading = phase { return } + guard let session = chat.session, let server = chat.serverID(of: record), let uri = record.app?.resourceURI, // ← SDK + record.mcpResult != nil else { return } // ← SDK + phase = .loading + let resource: MCPAppResource // ← SDK (MCPAppsHost) + var viewOnly = false + var changed = false + if chat.widgetCache.sha256(forRecord: record.id) == nil { // ← SDK (MCPAppsHost) + // First showing of this call: the server's current widget, kept and bound to the call. + do { + resource = try await chat.widgetCache.fetch(resourceURI: uri, server: server, manager: chat.manager, recordID: record.id) // ← SDK (MCPAppsHost) + } catch { + phase = .unavailable("Couldn't load the view (\(error.localizedDescription)).") + return + } + } else { + let check = await MCPAppRecreation.revalidate( // ← SDK (MCPAppsHost) + record, source: MCPServerManagerWidgetSource(manager: chat.manager, server: server), cache: chat.widgetCache) // ← SDK (MCPAppsHost) + switch check?.decision { + case .render(let offline)?: + guard let current = check?.resource else { phase = .unavailable("The view is no longer available."); return } + resource = current + viewOnly = offline + changed = check?.widgetChanged ?? false + case .needsRefresh(let automatic)?: + phase = .needsRefresh(automatic: automatic) + return + case .notApproved?: + phase = .unavailable("This version of the view isn't approved.") + return + case .unavailable?, nil: + phase = .unavailable("The server can't be reached and this view isn't saved.") + return + } + } + + let instance = record.id + let controller = pool.controller(for: instance) { // ← SDK (MCPAppsHost) + var actions = MCPAppsHostActions.session(session, instance: instance, openLink: { NSWorkspace.shared.open($0) }) // ← SDK (MCPAppsHost) + // Route widget messages through the chat so the turn shows as running and is saved. + actions.sendMessage = { [weak chat] text in chat?.sendFromWidget(text, instance: instance) } + return MCPAppViewController( // ← SDK (MCPAppsHost) + resource: resource, + tools: chat.manager.servers[server]?.tools ?? record.toolDescriptor.map { [$0] } ?? [], // ← SDK + backend: viewOnly ? OfflineBackend() as any MCPAppsBackend // ← SDK (MCPAppsHost) + : MCPAppsSessionBackend(session: session, manager: chat.manager, server: server, instance: instance), // ← SDK (MCPAppsHost) + configuration: MCPAppsBridgeConfiguration( // ← SDK (MCPAppsHost) + hostName: "MCP Chat", hostVersion: "0.1", + hostContext: ["displayMode": .string("inline"), "locale": .string(Locale.current.identifier(.bcp47))]), + bridgePolicy: ChatWidgetPolicy(viewOnly: viewOnly), + sandboxPolicy: .closed, + actions: actions) + } + MCPAppRecreation.deliver(record, to: controller) // ← SDK (MCPAppsHost) + phase = .live(controller, viewOnly: viewOnly, changed: changed) + await controller.load() // ← SDK (MCPAppsHost) + } + + /// Re-runs the call (as the host, through the session's authorizer). The fresh result arrives + /// as a new call at the end of the conversation, with the current view. + private func refresh() async { + guard let session = chat.session, let server = chat.serverID(of: record), case .mcp(_, let tool) = record.tool else { return } // ← SDK + refreshing = true + defer { refreshing = false } + let arguments = MCPAppRecreation.toolInput(from: record.arguments) // ← SDK (MCPAppsHost) + if case .failure(let error) = await session.callMCPTool(server: server, tool: tool, arguments: arguments, initiator: .host) { // ← SDK + chat.errorMessage = "Refresh failed: \(error)" + } + } +} + +/// Observes the controller so the slot follows the widget's reported height. +private struct LiveWidget: View { + @ObservedObject var controller: MCPAppViewController // ← SDK (MCPAppsHost) + + var body: some View { + MCPAppView(controller: controller) // ← SDK (MCPAppsHost) + .frame(height: max(80, controller.contentHeight ?? 360)) // ← SDK (MCPAppsHost) + .clipShape(RoundedRectangle(cornerRadius: 8)) + .overlay(RoundedRectangle(cornerRadius: 8).strokeBorder(.quaternary)) + } +} + +/// What a widget may ask the host for, on top of the protocol's own rules. Tool calls are allowed +/// here because the session's authorizer asks the user; messages and model context follow Settings. +struct ChatWidgetPolicy: MCPAppsBridgePolicy { // ← SDK (MCPAppsHost) + let viewOnly: Bool + + func authorizeToolCall(name: String, arguments: [String: MCPValue]) async -> MCPAppsDecision { // ← SDK (MCPAppsHost) + viewOnly ? .deny("Offline: this view is view-only") : .allow + } + + func authorizeOpenLink(_ url: URL) async -> MCPAppsDecision { // ← SDK (MCPAppsHost) + url.scheme?.lowercased() == "https" ? .allow : .deny("Only https links may be opened") + } + + func authorizeMessage(text: String) async -> MCPAppsDecision { // ← SDK (MCPAppsHost) + ChatSettings.allowWidgetMessages ? .allow : .deny("Widget messages are off in Settings") + } + + func authorizeModelContextUpdate() async -> MCPAppsDecision { // ← SDK (MCPAppsHost) + ChatSettings.allowWidgetContext ? .allow : .deny("Widget context is off in Settings") + } +} + +/// The backend for a view-only (offline) widget: nothing reaches the server. +struct OfflineBackend: MCPAppsBackend { // ← SDK (MCPAppsHost) + func callTool(name: String, arguments: [String: MCPValue]) async -> Result { .failure(.notConnected) } // ← SDK + func readResource(uri: String) async -> Result { .failure(.notConnected) } // ← SDK +} +``` + +### `Sources/MCPChat/ChatViews.swift` (excerpt) + +The conversation is a `ForEach` over `hostTranscript.entries` plus `replyInProgress`; the server +list and the approval sheet are `Components`. + +```swift +import LocalLMLabSDKComponents // ← Components +import LocalLMLabSDKCore // ← SDK +import LocalLMLabSDKMCPAppsHost // ← SDK (MCPAppsHost) +import SwiftUI + +struct ChatRootView: View { + @ObservedObject var chat: ChatModel + + var body: some View { + VStack(spacing: 0) { + if let session = chat.session { + ConversationView(chat: chat, transcript: session.hostTranscript) // ← SDK + } else { + ModelSetupView(chat: chat) + } + ForEach(chat.serversWithNoToolsOn, id: \.id) { server in + HStack(spacing: 8) { + Image(systemName: "exclamationmark.circle").foregroundStyle(.orange) + Text("\(server.displayName) is connected, but none of its tools are on, so the model can't use it.") + Spacer() + Button("Choose Tools…") { chat.showServers = true } + } + .font(.callout) + .padding(.horizontal, 12).padding(.vertical, 8) + .background(.orange.opacity(0.1)) + } + Divider() + Composer(chat: chat) + } + .frame(minWidth: 640, minHeight: 560) + .toolbar { + ToolbarItem(placement: .navigation) { + Text(chat.modelChoice.title).font(.callout).foregroundStyle(.secondary) + } + ToolbarItemGroup { + ToolsBadge(chat: chat) + Button { chat.showServers = true } label: { Label("MCP Servers", systemImage: "server.rack") } + Button { chat.newConversation() } label: { Label("New Chat", systemImage: "square.and.pencil") } + .disabled(chat.status == .responding) + } + } + .sheet(isPresented: $chat.showServers) { + VStack(alignment: .trailing) { + MCPServerPickerView(manager: chat.servers) // ← Components + Button("Done") { chat.showServers = false }.keyboardShortcut(.defaultAction) + } + .padding() + .frame(minWidth: 560, minHeight: 480) + } + .toolConfirmationSheet(chat.toolConfirm) // ← Components + .alert("Something went wrong", isPresented: Binding(get: { chat.errorMessage != nil }, set: { if !$0 { chat.errorMessage = nil } })) { + Button("OK") { chat.errorMessage = nil } + } message: { + Text(chat.errorMessage ?? "") + } + .task { await chat.start() } + } +} + +// … ModelSetupView — the first-run "download the model" screen (plain SwiftUI, omitted) + +private struct ConversationView: View { + @ObservedObject var chat: ChatModel + let transcript: HostTranscript // ← SDK + @State private var position = ScrollPosition(edge: .bottom) + + var body: some View { + ScrollView { + LazyVStack(alignment: .leading, spacing: 12) { + if transcript.entries.isEmpty { // ← SDK + Text("Ask something. Try “What's due today?” with Todoist connected.") + .foregroundStyle(.secondary).frame(maxWidth: .infinity).padding(.top, 60) + } + ForEach(transcript.entries) { entry in // ← SDK + EntryView(entry: entry, chat: chat) + .id(entry.id) + if let boundary = chat.memoryBoundaryTurn, entry.id == lastID(ofTurn: boundary) { + MemoryDivider() + } + } + if let reply = transcript.replyInProgress { // ← SDK + ReplyInProgressView(reply: reply) + } else if chat.status == .responding { + HStack(spacing: 8) { ProgressView().controlSize(.small); Text("Thinking…").foregroundStyle(.secondary) } + } + } + .padding(.horizontal, 16) + .padding(.top, 16) + // Room below the newest item, so it never sits against the message box. + .padding(.bottom, 72) + } + .scrollPosition($position) + .defaultScrollAnchor(.bottom) + // Follow the conversation while the user is at the bottom: whatever makes it taller — a new + // message, the spinner, streaming text, a widget loading or resizing — scrolls it into view. + // Someone who scrolled up to read stays where they are. + .onScrollGeometryChange(for: Layout.self) { geometry in + Layout(contentHeight: geometry.contentSize.height, + distanceFromBottom: geometry.contentSize.height - geometry.contentOffset.y - geometry.containerSize.height) + } action: { old, new in + if new.contentHeight != old.contentHeight, old.distanceFromBottom < 120 { + position.scrollTo(edge: .bottom) + } + } + // Sending a message always brings the bottom into view. + .onChange(of: chat.status) { _, status in + if status == .responding { withAnimation { position.scrollTo(edge: .bottom) } } + } + } + + private struct Layout: Equatable { + var contentHeight: CGFloat + var distanceFromBottom: CGFloat + } + + private func lastID(ofTurn turn: Int) -> String? { + transcript.entries.last { $0.turn == turn }?.id // ← SDK + } +} + +// … MemoryDivider — "the model doesn't remember the messages above" (plain SwiftUI, omitted) + +private struct EntryView: View { + let entry: HostTranscript.Entry // ← SDK + @ObservedObject var chat: ChatModel + + var body: some View { + switch entry.content { // ← SDK + case .user(let text): + UserBubble(text: text, fromWidget: entry.appInstance != nil) // ← SDK + case .assistant(let text): + AssistantMessage(text: text) + case .reasoning(let text): + ReasoningDisclosure(text: text) + case .toolCall(let record): // ← SDK + ToolCallView(record: record, chat: chat) + case .appContext(_, let text): // ← SDK + DisclosureGroup { + Text(text).font(.caption.monospaced()).textSelection(.enabled).frame(maxWidth: .infinity, alignment: .leading) + } label: { + Label("A view shared context with the model", systemImage: "rectangle.and.text.magnifyingglass") + .font(.caption).foregroundStyle(.secondary) + } + @unknown default: + EmptyView() + } + } +} + +// … UserBubble — a right-aligned bubble ("Sent by a view" when appInstance is set) (plain SwiftUI, omitted) + +// … AssistantMessage — the answer as Markdown (plain SwiftUI, omitted) + +private struct ReplyInProgressView: View { + let reply: HostTranscript.ReplyInProgress // ← SDK + + var body: some View { + VStack(alignment: .leading, spacing: 6) { + if let reasoning = reply.reasoning { ReasoningDisclosure(text: reasoning, thinking: reply.isReasoning) } // ← SDK + if reply.text.isEmpty { // ← SDK + if reply.reasoning == nil { // ← SDK + HStack(spacing: 8) { ProgressView().controlSize(.small); Text("Thinking…").foregroundStyle(.secondary) } + } + } else { + AssistantMessage(text: reply.text) // ← SDK + } + } + } +} + +/// The model's reasoning, collapsed by default. +// … ReasoningDisclosure — a collapsed "Reasoning" / "Thinking…" section (plain SwiftUI, omitted) + +private struct ToolCallView: View { + let record: ToolCallRecord // ← SDK + @ObservedObject var chat: ChatModel + + var body: some View { + VStack(alignment: .leading, spacing: 6) { + DisclosureGroup { + VStack(alignment: .leading, spacing: 4) { + Text("Arguments").font(.caption2).foregroundStyle(.secondary) + Text(record.argumentsDescription).font(.caption.monospaced()).textSelection(.enabled) // ← SDK + if let output = record.output { // ← SDK + Text("Result").font(.caption2).foregroundStyle(.secondary) + Text(output.prefix(2000)).font(.caption.monospaced()).textSelection(.enabled) + } + } + .frame(maxWidth: .infinity, alignment: .leading) + } label: { + HStack(spacing: 6) { + stateIcon + Text(record.tool.name).font(.callout.monospaced()) // ← SDK + if let by = initiatorLabel { Text(by).font(.caption2).foregroundStyle(.secondary) } + if case .denied(let reason) = record.state { Text("Not allowed: \(reason)").font(.caption).foregroundStyle(.secondary) } // ← SDK + if case .failed(let reason) = record.state { Text(reason).font(.caption).foregroundStyle(.red).lineLimit(1) } // ← SDK + } + } + if record.app != nil, record.state == .finished, record.mcpResult.map({ !$0.isError }) ?? false { // ← SDK + WidgetSlot(record: record, chat: chat, pool: chat.pool) // ← SDK (MCPAppsHost) + } + } + } + + private var initiatorLabel: String? { + switch record.initiator { // ← SDK + case .model: nil + case .app: "by a view" + case .host: "refresh" + @unknown default: nil + } + } + + @ViewBuilder private var stateIcon: some View { + switch record.state { // ← SDK + case .awaitingApproval: Image(systemName: "hand.raised").foregroundStyle(.orange) + case .running: ProgressView().controlSize(.mini) + case .finished: Image(systemName: "checkmark.circle").foregroundStyle(.green) + case .failed: Image(systemName: "exclamationmark.triangle").foregroundStyle(.red) + case .denied: Image(systemName: "nosign").foregroundStyle(.secondary) + @unknown default: Image(systemName: "questionmark.circle") + } + } +} + +// … Composer — the message field (plain SwiftUI, omitted) + +// … SettingsView — @AppStorage toggles for the ChatSettings keys (plain SwiftUI, omitted) + +private struct ToolsBadge: View { + @ObservedObject var chat: ChatModel + @ObservedObject private var servers: MCPServerManagerObservable // ← Components + @State private var showList = false + + init(chat: ChatModel) { + self.chat = chat + self.servers = chat.servers + } + + var body: some View { + let tools = chat.modelTools // ← SDK + let builtIn = chat.builtInToolNames + let count = tools.count + builtIn.count + Button { showList.toggle() } label: { + Label("\(count) tool\(count == 1 ? "" : "s")", systemImage: "wrench.and.screwdriver") + .labelStyle(.titleAndIcon) + } + .help("MCP tools the model can use") + .popover(isPresented: $showList) { + VStack(alignment: .leading, spacing: 6) { + Text("Tools the model can use").font(.headline) + ForEach(builtIn, id: \.self) { name in + HStack(spacing: 6) { + Text(name).font(.callout.monospaced()) + Text("built in").font(.caption2).foregroundStyle(.secondary) + } + } + if tools.isEmpty { + Text("No MCP tools. Turn tools on in the server list.").foregroundStyle(.secondary) + } + ForEach(tools, id: \.name) { tool in + HStack(spacing: 6) { + Text(tool.name).font(.callout.monospaced()) + if tool.app != nil { Image(systemName: "rectangle.on.rectangle").help("Shows an interactive view") } // ← SDK + } + } + Button("Server List…") { showList = false; chat.showServers = true }.padding(.top, 4) + } + .padding(14).frame(minWidth: 260, alignment: .leading) + } + } +} +``` + +### `Sources/MCPChat/MCPChatApp.swift` + +```swift +import AppKit +import LocalLMLabSDKCore // ← SDK +import SwiftUI + +// OAuth callbacks arrive as a custom-scheme URL. Route them through the AppDelegate, not SwiftUI's +// .onOpenURL — WindowGroup opens an extra window per open-URL event. +private final class AppDelegate: NSObject, NSApplicationDelegate { + func application(_ application: NSApplication, open urls: [URL]) { + for url in urls { MCPOAuthRedirectListener.shared.handleRedirect(url) } // ← SDK + } + + func applicationShouldTerminateAfterLastWindowClosed(_ sender: NSApplication) -> Bool { true } +} + +@main +struct MCPChatApp: App { + @NSApplicationDelegateAdaptor(AppDelegate.self) private var appDelegate + @StateObject private var chat: ChatModel + + init() { + // Distinct scheme (see Info.plist) so this app's OAuth callback can't reach another app. + MCPOAuthFlow.redirectURI = "mcpchat://oauth/callback" // ← SDK + _chat = StateObject(wrappedValue: ChatModel()) + } + + var body: some Scene { + WindowGroup { + ChatRootView(chat: chat) + } + .handlesExternalEvents(matching: []) + .defaultSize(width: 820, height: 760) + .commands { + CommandGroup(replacing: .newItem) { + Button("New Chat") { chat.newConversation() }.keyboardShortcut("n") + } + } + + Settings { + SettingsView() + } + } +} +``` + +**Tally**: 139 lines carry a marker — more than any other example, because this one is mostly SDK +surface: 86 `← SDK` (setup, sessions, the transcript the UI draws, saving and restoring servers +and conversations), 39 `← SDK (MCPAppsHost)` (rendering, the pool, the cache, re-creation, the +bridge policy), 4 `← SDK (Inference)` (the pinned Qwen3 provider and its download), 10 +`← Components` (the server list, the approval sheet, the observable server state). What the app writes itself is the chat look, +the settings, where conversations and servers are saved, and the policy a view runs under +(`ChatWidgetPolicy`: tool calls allowed because the session's authorizer asks; messages and +shared context per Settings; https links only). From 21253aa8bd4a0d7119ad5c7d65054c471f68d4f1 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 18:55:18 -0700 Subject: [PATCH 17/73] =?UTF-8?q?docs:=20api-surface=20from=20the=20publis?= =?UTF-8?q?hed=20v2.0.0-dev=20binaries;=20lab.mcp=20note=20in=20=C2=A76?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - api-surface.md regenerated from the v2.0.0-dev release xcframeworks (was still 1.0.0-GA): 2.0 folds, per-server trust/approval, chat-host and MCP Apps APIs, MCP 2026-07-28; MCPAppsHost listed like Components. - sdk-guide.md §6: an app that uses LocalLMLab uses lab.mcp, not its own MCPServerManager. Co-Authored-By: Claude --- docs/api-surface.md | 448 ++++++++++++++++++++++++++++++++++++++++---- docs/sdk-guide.md | 5 + 2 files changed, 421 insertions(+), 32 deletions(-) diff --git a/docs/api-surface.md b/docs/api-surface.md index a9965c4..6b3df8a 100644 --- a/docs/api-surface.md +++ b/docs/api-surface.md @@ -2,7 +2,7 @@ > Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s. > Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check. -> SDK version: `"1.0.0-GA"` · generated 2026-09-27T18:00Z +> SDK version: `"2.0.0-dev"` (the published `v2.0.0-dev` xcframeworks) · generated 2026-10-03T01:54Z ## LocalLMLabSDKCore @@ -638,15 +638,48 @@ public typealias Output = String public enum GenerationErrorDescription public static func describe(_ error: any Error) async -> String public static func describeSync(_ error: any Error) -> String +public enum ArchiveError : Error, Equatable +case notEmpty +case unsupportedFormat(String) +public static func == (a: HostTranscript.ArchiveError, b: HostTranscript.ArchiveError) -> Bool +@MainActor final public func archive() throws -> Data +@MainActor final public func restore(from data: Data) throws +@MainActor final public class HostTranscript +public struct Entry : Identifiable, Sendable +public enum Content : Sendable +case user(String) +case assistant(String) +case reasoning(String) +case toolCall(ToolCallRecord) +case appContext(instance: String, text: String) +public let id: String +public var content: HostTranscript.Entry.Content +public let turn: Int +public var appInstance: String? +public typealias ID = String +@MainActor final public var entries: [HostTranscript.Entry] +@MainActor final public var replyInProgress: HostTranscript.ReplyInProgress? +public struct ReplyInProgress : Sendable, Equatable +public var reasoning: String? +public var text: String +public var isReasoning: Bool +public init(reasoning: String?, text: String, isReasoning: Bool) +public static func == (a: HostTranscript.ReplyInProgress, b: HostTranscript.ReplyInProgress) -> Bool +@MainActor final public func appOnlyCallCount(for instance: String) -> Int +@MainActor final public var resultMemoryBudget: Int +@MainActor final public var spillDirectory: URL? +@MainActor final public var toolCalls: [ToolCallRecord] +@MainActor final public func toolCall(id: String) -> ToolCallRecord? public enum JSONPath public static func value(at path: String, in root: Any) -> Any? public static func stringify(_ value: Any?) -> String -@MainActor final public func makeSession(route: RouteName, tools: [any Tool] = [], instructions: String? = nil, includeMCPTools: Bool = true, options: SessionOptions = .init(), authorizer: (any ToolCallAuthorizer)? = nil) throws -> LocalLMLabSession +@MainActor final public func makeSession(route: RouteName, tools: [any Tool] = [], instructions: String? = nil, restoring transcript: Transcript? = nil, includeMCPTools: Bool = true, mcpAppHints: Bool = false, options: SessionOptions = .init(), authorizer: (any ToolCallAuthorizer)? = nil) throws -> LocalLMLabSession @MainActor final public class LocalLMLab public struct Configuration public var providers: [any ModelProvider] public var state: LocalLMLabState? -public init(providers: [any ModelProvider] = [], state: LocalLMLabState? = nil) +public var mcp: MCPSettings +public init(providers: [any ModelProvider] = [], state: LocalLMLabState? = nil, mcp: MCPSettings = MCPSettings()) @MainActor final public let models: ModelRegistry @MainActor final public let mcp: MCPServerManager @MainActor final public let connectors: ConnectorsFacade @@ -654,6 +687,12 @@ public init(providers: [any ModelProvider] = [], state: LocalLMLabState? = nil) @MainActor public init(configuration: LocalLMLab.Configuration = .init()) @MainActor final public func snapshot() -> LocalLMLabState @MainActor final public func restore(from state: LocalLMLabState) +public struct MCPSettings +public var handlers: MCPClientHandlers +public var responseLimits: MCPResponseLimits +public var versionNegotiation: MCPVersionNegotiation +public var liveUpdates: Bool +public init(handlers: MCPClientHandlers = MCPClientHandlers(), responseLimits: MCPResponseLimits = .default, versionNegotiation: MCPVersionNegotiation = .auto, liveUpdates: Bool = true) public struct LocalLMLabState : Codable, Sendable, Equatable public static let currentVersion: Int public var version: Int @@ -681,15 +720,19 @@ case connector(detail: String) public var errorDescription: String? public enum LocalLMLabSDKVersion public static let current: String -public func respond(to prompt: String) async throws -> String -public func respond(to prompt: String, options: SessionOptions) async throws -> String +@MainActor public func callMCPTool(server: MCPServerID, tool: String, arguments: [String : MCPValue], initiator: ToolCallInitiator) async -> Result +@MainActor public func updateModelContext(_ text: String, fromAppInstance instance: String) +public func respond(to prompt: String, options: SessionOptions = SessionOptions(), fromAppInstance instance: String? = nil) async throws -> String +public func streamResponse(to prompt: String, options: SessionOptions = SessionOptions(), fromAppInstance instance: String? = nil) -> AsyncThrowingStream public struct LocalLMLabSession : Sendable public let route: RouteName public let modelID: ModelID public var retryOnContextOverflow: RetryPolicy +public var turnContext: (@Sendable () -> String?)? public var languageModelSession: LanguageModelSession public var events: AsyncStream public var citations: [Citation] +public var hostTranscript: HostTranscript public func reconcileServerToolArtifacts() public var contextBudget: ContextBudget public func cancel() @@ -729,6 +772,25 @@ public typealias ID = GenerationID nonisolated public func call(arguments: GetCurrentLocationTool.Arguments) async throws -> String public typealias Output = String nonisolated public init(_ content: GeneratedContent) throws +public enum MCPAppVisibility : String, Sendable, Codable, Hashable, CaseIterable +case model +case app +public init?(rawValue: String) +public typealias AllCases = [MCPAppVisibility] +nonisolated public static var allCases: [MCPAppVisibility] +public var rawValue: String +public struct MCPAppLink : Sendable, Equatable, Hashable +public var resourceURI: String +public var visibility: Set +public init(resourceURI: String, visibility: Set) +public static func == (a: MCPAppLink, b: MCPAppLink) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public var app: MCPAppLink? +public var appVisibility: Set +public var isModelVisible: Bool +public var isMCPAppResource: Bool +public var isMCPAppResource: Bool public struct MCPElicitationField : Sendable, Equatable public enum Kind : Sendable, Equatable case string(format: String?) @@ -790,7 +852,8 @@ public var elicitation: (any MCPElicitationHandler)? public var sampling: (any MCPSamplingHandler)? public var roots: (any MCPRootsProvider)? public var logging: (any MCPLoggingSink)? -public init(elicitation: (any MCPElicitationHandler)? = nil, sampling: (any MCPSamplingHandler)? = nil, roots: (any MCPRootsProvider)? = nil, logging: (any MCPLoggingSink)? = nil) +public var extensions: [String : MCPValue] +public init(elicitation: (any MCPElicitationHandler)? = nil, sampling: (any MCPSamplingHandler)? = nil, roots: (any MCPRootsProvider)? = nil, logging: (any MCPLoggingSink)? = nil, extensions: [String : MCPValue] = [:]) public enum MCPStoredCredentialKind : String, Sendable, Equatable case oauth case pat @@ -833,19 +896,26 @@ nonisolated(unsafe) public static var clientMetadataURL: URL? @MainActor final public class MCPOAuthRedirectListener @MainActor public static let shared: MCPOAuthRedirectListener @MainActor final public func handleRedirect(_ url: URL) +@MainActor final public func cancelAll() public enum MCPProtocolVersion : String, CaseIterable, Comparable, Sendable, Codable case v2024_11_05 case v2025_03_26 case v2025_06_18 case v2025_11_25 +case v2026_07_28 public static let clientPreferred: MCPProtocolVersion +public static let minimumSupported: MCPProtocolVersion +public static var allCases: [MCPProtocolVersion] +public static let statelessPreferred: MCPProtocolVersion +public var isStateless: Bool public static func < (lhs: MCPProtocolVersion, rhs: MCPProtocolVersion) -> Bool public var supportsStructuredContent: Bool public var supportsElicitation: Bool public var supportsIconsAndRicherElicitation: Bool +public var sendsProtocolVersionHeader: Bool +public var supportsURLElicitation: Bool public init?(rawValue: String) public typealias AllCases = [MCPProtocolVersion] -nonisolated public static var allCases: [MCPProtocolVersion] public var rawValue: String public struct MCPServerCapabilities : Sendable, Codable, Equatable public var tools: Bool @@ -877,16 +947,28 @@ public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws public enum MCPConnectionMode : Sendable, Equatable case handshake(MCPProtocolVersion) +case stateless(MCPProtocolVersion) public var negotiatedVersion: MCPProtocolVersion +public var isStateless: Bool public static func == (a: MCPConnectionMode, b: MCPConnectionMode) -> Bool +public enum MCPVersionNegotiation : Sendable, Equatable +case auto +case legacy +case pin(MCPProtocolVersion) +public static func == (a: MCPVersionNegotiation, b: MCPVersionNegotiation) -> Bool @MainActor final public class MCPServerManager @MainActor final public var servers: [MCPServerID : MCPServerState] @MainActor final public var serverChanges: AsyncStream<[MCPServerID : MCPServerState]> -@MainActor public init(responseLimits: MCPResponseLimits = .default, handlers: MCPClientHandlers = MCPClientHandlers()) +@MainActor public init(responseLimits: MCPResponseLimits = .default, handlers: MCPClientHandlers = MCPClientHandlers(), versionNegotiation: MCPVersionNegotiation = .auto, liveUpdates: Bool = true) @MainActor final public func addServer(url: URL, displayName: String, authType: MCPAuthType = .none, patToken: String? = nil, manualClientID: String? = nil) async -> Result @MainActor final public func reconnect(_ id: MCPServerID) async -> Result @MainActor final public func disconnect(_ id: MCPServerID) @MainActor final public func toolsForSession() -> [MCPToolDescriptor] +@MainActor final public func setTrust(_ trust: MCPServerTrust, server: MCPServerID) +@MainActor final public func setToolApproval(_ approval: ToolApproval?, server: MCPServerID) +@MainActor final public func toolApproval(server: MCPServerID) -> ToolApproval +@MainActor final public func setFileBackedOutput(_ output: MCPFileBackedOutput?, server: MCPServerID) +@MainActor final public func fileBackedOutput(server: MCPServerID) -> MCPFileBackedOutput? @MainActor final public func setToolEnabled(server: MCPServerID, tool: String, enabled: Bool) @MainActor final public func callTool(server: MCPServerID, tool: String, arguments: [String : MCPValue], allowElicitation: Bool = true) async -> Result @MainActor final public func resourcesForSession() -> [MCPResourceDescriptor] @@ -923,6 +1005,11 @@ public var resourceTemplates: [MCPResourceTemplateDescriptor] public var prompts: [MCPPromptDescriptor] public var negotiatedProtocolVersion: MCPProtocolVersion? public var serverInstructions: String? +public var liveUpdates: MCPLiveUpdates? +public var trust: MCPServerTrust +public var toolApproval: ToolApproval? +public var trustedToolsDigest: String? +public var toolsChangedSinceTrusted: Bool public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws public func exportSummary() -> String @@ -947,7 +1034,8 @@ public var description: String public var mimeType: String? public var enabled: Bool public var title: String? -public init(serverID: MCPServerID, uri: String, name: String, description: String, mimeType: String? = nil, enabled: Bool = true, title: String? = nil) +public var meta: MCPValue? +public init(serverID: MCPServerID, uri: String, name: String, description: String, mimeType: String? = nil, enabled: Bool = true, title: String? = nil, meta: MCPValue? = nil) public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws public struct MCPResourceTemplateDescriptor : Codable, Sendable @@ -978,6 +1066,8 @@ public var uri: String public var mimeType: String? public var text: String? public var blob: String? +public var meta: MCPValue? +public init(uri: String, mimeType: String? = nil, text: String? = nil, blob: String? = nil, meta: MCPValue? = nil) public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws public struct MCPPromptMessage : Codable, Sendable @@ -985,6 +1075,18 @@ public var role: String public var text: String public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws +public struct MCPToolAnnotations : Codable, Sendable, Equatable +public var title: String? +public var readOnlyHint: Bool? +public var destructiveHint: Bool? +public var idempotentHint: Bool? +public var openWorldHint: Bool? +public init(title: String? = nil, readOnlyHint: Bool? = nil, destructiveHint: Bool? = nil, idempotentHint: Bool? = nil, openWorldHint: Bool? = nil) +public var isReadOnly: Bool +public var isDestructive: Bool +public static func == (a: MCPToolAnnotations, b: MCPToolAnnotations) -> Bool +public func encode(to encoder: any Encoder) throws +public init(from decoder: any Decoder) throws public struct MCPToolDescriptor : Codable, Sendable public var serverID: MCPServerID public var name: String @@ -994,7 +1096,9 @@ public var estimatedTokens: Int public var enabled: Bool public var title: String? public var outputSchema: Data? -public init(serverID: MCPServerID, name: String, description: String, rawSchema: Data, estimatedTokens: Int, enabled: Bool = true, title: String? = nil, outputSchema: Data? = nil) +public var meta: MCPValue? +public var annotations: MCPToolAnnotations? +public init(serverID: MCPServerID, name: String, description: String, rawSchema: Data, estimatedTokens: Int, enabled: Bool = true, title: String? = nil, outputSchema: Data? = nil, meta: MCPValue? = nil, annotations: MCPToolAnnotations? = nil) public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws indirect public enum MCPValue : Codable, Sendable, Equatable @@ -1023,6 +1127,38 @@ case responseTooLarge(String) public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws public protocol MCPConnection : Sendable +public struct MCPSubscriptionFilter : Sendable, Codable, Equatable +public var toolsListChanged: Bool +public var promptsListChanged: Bool +public var resourcesListChanged: Bool +public init(toolsListChanged: Bool = false, promptsListChanged: Bool = false, resourcesListChanged: Bool = false) +public var isEmpty: Bool +public static func == (a: MCPSubscriptionFilter, b: MCPSubscriptionFilter) -> Bool +public func encode(to encoder: any Encoder) throws +public init(from decoder: any Decoder) throws +public enum MCPChangeEvent : Sendable, Equatable +case acknowledged(MCPSubscriptionFilter) +case toolsChanged +case promptsChanged +case resourcesChanged +case ended(MCPListenEnd) +public static func == (a: MCPChangeEvent, b: MCPChangeEvent) -> Bool +public enum MCPListenEnd : Sendable, Equatable +case closed +case failed(String) +case refused(String) +case unauthorized +public static func == (a: MCPListenEnd, b: MCPListenEnd) -> Bool +public enum MCPLiveUpdates : Sendable, Codable, Equatable +case active(MCPSubscriptionFilter) +case reconnecting +case unavailable(String) +public static func == (a: MCPLiveUpdates, b: MCPLiveUpdates) -> Bool +public func encode(to encoder: any Encoder) throws +public init(from decoder: any Decoder) throws +public func listen(filter: MCPSubscriptionFilter) -> AsyncStream +public func refreshAuthorizationSilently() async -> Bool +public var authorizationGeneration: Int public enum MCPToolAdapterError : Error, CustomStringConvertible case invalidRawSchema case unbuildableSchema(String) @@ -1035,6 +1171,13 @@ public init(descriptor: MCPToolDescriptor, manager: MCPServerManager) throws nonisolated public func call(arguments: GeneratedContent) async throws -> String public typealias Arguments = GeneratedContent public typealias Output = String +public struct MCPFileBackedOutput : Sendable, Equatable +public var root: URL +public var previewCharacters: Int +public var inlineCharacterLimit: Int? +public var followUp: String +public init(root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, followUp: String = "read it back a window at a time with readFileRange") +public static func == (a: MCPFileBackedOutput, b: MCPFileBackedOutput) -> Bool public static func mcp(descriptor: MCPToolDescriptor, manager: MCPServerManager, root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, followUp: String = "read it back a window at a time with readFileRange") throws -> FileBackedTool public struct MCPResourceLink : Sendable, Codable, Equatable public var uri: String @@ -1052,7 +1195,8 @@ public var structuredContent: MCPValue? public var resourceLinks: [MCPResourceLink] public var isError: Bool public var truncated: Bool -public init(text: String = "", structuredContent: MCPValue? = nil, resourceLinks: [MCPResourceLink] = [], isError: Bool = false, truncated: Bool = false) +public var meta: MCPValue? +public init(text: String = "", structuredContent: MCPValue? = nil, resourceLinks: [MCPResourceLink] = [], isError: Bool = false, truncated: Bool = false, meta: MCPValue? = nil) public var renderedForModel: String public var rawPayload: String public static func == (a: MCPToolResult, b: MCPToolResult) -> Bool @@ -1064,7 +1208,8 @@ public var maxDescriptorCount: Int public var maxSchemaBytes: Int public var maxTextFieldLength: Int public var maxInlineContentBytes: Int -public init(maxResponseBytes: Int = 8 * 1024 * 1024, maxDescriptorCount: Int = 512, maxSchemaBytes: Int = 128 * 1024, maxTextFieldLength: Int = 16 * 1024, maxInlineContentBytes: Int = 1 * 1024 * 1024) +public var maxUIResourceBytes: Int +public init(maxResponseBytes: Int = 8 * 1024 * 1024, maxDescriptorCount: Int = 512, maxSchemaBytes: Int = 128 * 1024, maxTextFieldLength: Int = 16 * 1024, maxInlineContentBytes: Int = 1 * 1024 * 1024, maxUIResourceBytes: Int = 4 * 1024 * 1024) public static let `default`: MCPResponseLimits public static func == (a: MCPResponseLimits, b: MCPResponseLimits) -> Bool public enum ModelAvailability : Sendable, Equatable @@ -1306,18 +1451,6 @@ public typealias StringLiteralType = String public typealias UnicodeScalarLiteralType = String public func hash(into hasher: inout Hasher) public var hashValue: Int -public struct RuleBasedToolAuthorizer : ToolCallAuthorizer -public enum Rule : Sendable -case denyTool(String) -case denyMCPTools -case deny(atOrAbove: ToolImpact) -case allowTool(String) -case confirm(atOrAbove: ToolImpact) -case confirmMCPTools -public init(rules: [RuleBasedToolAuthorizer.Rule], denyReason: String = "blocked by this app's tool policy", confirm: @escaping @Sendable (PendingToolCall) async -> Bool = { _ in false }) -public func authorize(_ call: PendingToolCall) async -> ToolCallDecision -public static func confirmingMutations(confirm: @escaping @Sendable (PendingToolCall) async -> Bool) -> RuleBasedToolAuthorizer -public static func confirmingDestructive(confirm: @escaping @Sendable (PendingToolCall) async -> Bool) -> RuleBasedToolAuthorizer public struct LoadTableTool : Tool public static let defaultDescription: String public let name: String @@ -1360,6 +1493,31 @@ nonisolated public func call(arguments a: SQLQueryTool.Arguments) async throws - public typealias Output = String nonisolated public init(_ content: GeneratedContent) throws nonisolated public init(_ content: GeneratedContent) throws +public enum MCPServerTrust : String, Sendable, Codable, CaseIterable +case untrusted +case trusted +public init?(rawValue: String) +public typealias AllCases = [MCPServerTrust] +nonisolated public static var allCases: [MCPServerTrust] +public var rawValue: String +public enum ToolApprover : String, Sendable, Codable +case user +case policy +public init?(rawValue: String) +public var rawValue: String +public enum ToolApproval : Sendable, Codable, Equatable +case allow +case ask(atOrAbove: ToolImpact, by: ToolApprover) +public static func `default`(for trust: MCPServerTrust) -> ToolApproval +public static func == (a: ToolApproval, b: ToolApproval) -> Bool +public func encode(to encoder: any Encoder) throws +public init(from decoder: any Decoder) throws +public typealias ToolPolicy = @Sendable (PendingToolCall) async -> ToolPolicyDecision +public enum ToolPolicyDecision : Sendable, Equatable +case allow +case deny(reason: String) +case askUser +public static func == (a: ToolPolicyDecision, b: ToolPolicyDecision) -> Bool public enum SessionEvent : Sendable case modelLoadProgress(Double) case routeSwitched(from: ModelID, to: ModelID) @@ -1461,17 +1619,21 @@ public let arguments: GeneratedContent? public let argumentsDescription: String public let origin: ToolOrigin public let impact: ToolImpact -public init(toolName: String, arguments: GeneratedContent?, argumentsDescription: String, origin: ToolOrigin, impact: ToolImpact) +public let initiator: ToolCallInitiator +public let serverApproval: ToolApproval? +public init(toolName: String, arguments: GeneratedContent?, argumentsDescription: String, origin: ToolOrigin, impact: ToolImpact, initiator: ToolCallInitiator = .model, serverApproval: ToolApproval? = nil) public var summary: PendingToolCallSummary public struct PendingToolCallSummary : Codable, Sendable, Equatable public let toolName: String public let argumentsDescription: String public let origin: ToolOrigin public let impact: ToolImpact -public init(toolName: String, argumentsDescription: String, origin: ToolOrigin, impact: ToolImpact) +public let initiator: ToolCallInitiator +public let serverApproval: ToolApproval? +public init(toolName: String, argumentsDescription: String, origin: ToolOrigin, impact: ToolImpact, initiator: ToolCallInitiator = .model, serverApproval: ToolApproval? = nil) +public init(from decoder: any Decoder) throws public static func == (a: PendingToolCallSummary, b: PendingToolCallSummary) -> Bool public func encode(to encoder: any Encoder) throws -public init(from decoder: any Decoder) throws public enum ToolOrigin : Sendable, Codable, Equatable case host case mcp(server: MCPServerID, displayName: String) @@ -1490,13 +1652,46 @@ public var rawValue: Int public protocol ImpactRatedTool : Tool public protocol OriginTaggedTool : Tool public func limited(toMaxImpact ceiling: ToolImpact) -> [any Tool] +public enum ToolCallInitiator : Sendable, Hashable, Codable +case model +case app(instance: String) +case host +public static func == (a: ToolCallInitiator, b: ToolCallInitiator) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public enum ToolReference : Sendable, Equatable +case mcp(server: MCPServerID, tool: String) +case host(name: String) +case provider(ServerToolActivity) +public var name: String +public static func == (a: ToolReference, b: ToolReference) -> Bool +public struct ToolCallRecord : Identifiable, Sendable +public enum State : Sendable, Equatable +case awaitingApproval +case running +case finished +case failed(reason: String) +case denied(reason: String) +public static func == (a: ToolCallRecord.State, b: ToolCallRecord.State) -> Bool +public let id: String +public var modelCallID: String? +public let tool: ToolReference +public let initiator: ToolCallInitiator +public let arguments: GeneratedContent? +public let argumentsDescription: String +public var state: ToolCallRecord.State +public var output: String? +public var mcpResult: MCPToolResult? +public let app: MCPAppLink? +public let toolDescriptor: MCPToolDescriptor? +public let startedAt: Date +public var finishedAt: Date? +public typealias ID = String public protocol ToolConfirmationChannel : Sendable public struct ConfirmingToolAuthorizer : ToolCallAuthorizer -public enum Requirement : Sendable -case allow -case confirm -case deny(reason: String) -public init(channel: any ToolConfirmationChannel, timeout: Duration = .seconds(120), requirement: @escaping @Sendable (PendingToolCall) -> ConfirmingToolAuthorizer.Requirement = { call in +public init(channel: (any ToolConfirmationChannel)?, policy: ToolPolicy? = nil, hostTools: ToolApproval = .ask(atOrAbove: .mutate, by: .user), timeout: Duration = .seconds(120)) public func authorize(_ call: PendingToolCall) async -> ToolCallDecision final public class DecisionGate : @unchecked Sendable public init() @@ -2207,3 +2402,192 @@ public var validates: Bool public var webSearchEnabled: Bool public var webSearchSupported: Bool ``` + +## LocalLMLabSDKMCPAppsHost + +> Ships as SOURCE, like Components — a grep of the public declarations in +> `MCPAppsHost/Sources/`, not a compiler-derived interface. + +```swift +public enum Direction: String, Sendable { case fromView, toView } +public enum Kind: Sendable, Equatable +public enum ListElement: Sendable, Equatable { case string, integer, number } +public enum LoadState: Equatable +public enum MCPAppRecreation +public enum MCPAppRecreationDecision: Sendable, Equatable +public enum MCPAppResourceError: Error, Equatable, CustomStringConvertible +public enum MCPAppsDecision: Sendable, Equatable +public enum MCPAppsSandbox +public enum MCPToolInputError: Error, Equatable, CustomStringConvertible +public enum Outcome: String, Sendable { case ok, denied, error, dropped } +public enum State: Sendable, Equatable +public final class MCPAppViewController: NSObject, ObservableObject +public final class MCPAppViewPool: ObservableObject +public final class MCPAppWidgetCache +public final class MCPAppsBridge +public func advertisingMCPApps() -> MCPClientHandlers +public func arguments(from values: [String: String]) -> Result<[String: MCPValue], MCPToolInputError> +public func authorizeMessage(text: String) async -> MCPAppsDecision { .deny("Message sending denied by policy") } +public func authorizeModelContextUpdate() async -> MCPAppsDecision { .deny("Context update denied by policy") } +public func authorizeOpenLink(_ url: URL) async -> MCPAppsDecision { .deny("Link opening denied by policy") } +public func authorizeResourceRead(uri: String) async -> MCPAppsDecision +public func authorizeToolCall(name: String, arguments: [String: MCPValue]) async -> MCPAppsDecision +public func authorizeToolCall(name: String, arguments: [String: MCPValue]) async -> MCPAppsDecision { .allow } +public func bind(recordID: String, sha256: String) +public func callTool(name: String, arguments: [String: MCPValue]) async -> Result +public func close() async +public func controller(for key: String, make: () throws -> MCPAppViewController) rethrows -> MCPAppViewController +public func currentDescriptor(tool: String) -> MCPToolDescriptor? +public func currentResource(uri: String) async -> Result +public func deliverHostContextChange(_ partial: [String: MCPValue]) +public func deliverToolCancelled(reason: String? = nil) +public func deliverToolInput(arguments: [String: MCPValue]) +public func deliverToolResult(_ result: MCPToolResult) +public func discardSnapshot(for key: String) { snapshots.removeValue(forKey: key) } +public func evaluateInWidget(_ script: String) async throws -> Any? +public func exitFullscreen() +public func fetch(resourceURI: String, server: MCPServerID, manager: MCPServerManager, +public func isLive(_ key: String) -> Bool { controllers[key] != nil } +public func load() async +public func makeNSView(context: Context) -> WKWebView { controller.webView } +public func readResource(uri: String) async -> Result +public func receive(_ data: Data) async +public func release(_ key: String) async +public func releaseAll() async +public func resource(sha256: String) -> MCPAppResource? +public func setDisplayMode(_ mode: String) +public func sha256(forRecord recordID: String) -> String? { index.recordVersion[recordID] } +public func snapshot(for key: String) -> NSImage? { snapshots[key] } +public func store(_ content: MCPResourceContent, requestedURI: String) throws -> MCPAppResource +public func teardown(reason: String = "host closing") async +public func touch(_ key: String) +public func updateNSView(_ nsView: WKWebView, context: Context) {} +public func webView(_ webView: WKWebView, createWebViewWith configuration: WKWebViewConfiguration, for navigationAction: WKNavigationAction, windowFeatures: WKWindowFeatures) -> WKWebView? +public func webView(_ webView: WKWebView, decidePolicyFor navigationAction: WKNavigationAction) async -> WKNavigationActionPolicy +public func webView(_ webView: WKWebView, didFail navigation: WKNavigation!, withError error: any Error) +public func webView(_ webView: WKWebView, didFailProvisionalNavigation navigation: WKNavigation!, withError error: any Error) +public func webView(_ webView: WKWebView, didFinish navigation: WKNavigation!) +public func webView(_ webView: WKWebView, requestMediaCapturePermissionFor origin: WKSecurityOrigin, initiatedByFrame frame: WKFrameInfo, type: WKMediaCaptureType, decisionHandler: @escaping @MainActor @Sendable (WKPermissionDecision) -> Void) +public func webView(_ webView: WKWebView, runJavaScriptAlertPanelWithMessage message: String, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable () -> Void) +public func webView(_ webView: WKWebView, runJavaScriptConfirmPanelWithMessage message: String, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable (Bool) -> Void) +public func webView(_ webView: WKWebView, runJavaScriptTextInputPanelWithPrompt prompt: String, defaultText: String?, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable (String?) -> Void) +public func webView(_ webView: WKWebView, runOpenPanelWith parameters: WKOpenPanelParameters, initiatedByFrame frame: WKFrameInfo, completionHandler: @escaping @MainActor @Sendable ([URL]?) -> Void) +public init( +public init(allowedTools: Set? = nil) +public init(camera: Bool = false, microphone: Bool = false, geolocation: Bool = false, clipboardWrite: Bool = false) +public init(connectDomains: [String] = [], resourceDomains: [String] = [], frameDomains: [String] = [], baseUriDomains: [String] = []) +public init(content: MCPResourceContent, requestedURI: String, listedMeta: MCPValue? = nil) throws +public init(controller: MCPAppViewController) +public init(csp: MCPAppCSP = MCPAppCSP(), permissions: MCPAppPermissions = MCPAppPermissions(), domain: String? = nil, prefersBorder: Bool? = nil) +public init(direction: Direction, method: String, outcome: Outcome, detail: String? = nil) +public init(directory: URL? = nil, maxBytes: Int = 64 * 1024 * 1024) +public init(limit: Int = 3) +public init(manager: MCPServerManager, server: MCPServerID) +public init(meta: MCPValue?) +public init(schema: Data) +public init(session: LocalLMLabSession, manager: MCPServerManager, server: MCPServerID, instance: String) +public let defaultText: String? +public let detail: String? +public let direction: Direction +public let directory: URL +public let effectiveCSP: MCPAppCSP +public let fields: [MCPToolInputField] +public let html: String +public let info: MCPAppResourceInfo +public let instance: String +public let isRequired: Bool +public let kind: Kind +public let manager: MCPServerManager +public let method: String +public let name: String +public let outcome: Outcome +public let resource: MCPAppResource +public let server: MCPServerID +public let session: LocalLMLabSession +public let sha256: String +public let summary: String +public let supportedDisplayModes: [String] +public let uri: String +public let webView: WKWebView +public private(set) var bridge: MCPAppsBridge! +public private(set) var displayMode: String +public private(set) var state: State = .awaitingInitialize +public protocol MCPAppWidgetSource +public protocol MCPAppsBackend +public protocol MCPAppsBridgePolicy +public static func cachedResource(recordID: String, cache: MCPAppWidgetCache) -> MCPAppResource? +public static func contentRuleListJSON(for csp: MCPAppCSP) -> String +public static func decide(cachedSHA256: String?, currentSHA256: String?, outputSchemaUnchanged: Bool, +public static func deliver(_ record: ToolCallRecord, to controller: MCPAppViewController) -> Bool +public static func deliver(arguments: GeneratedContent?, result: MCPToolResult, to controller: MCPAppViewController) +public static func effectiveCSP(declared: MCPAppCSP, policy: MCPAppsSandboxPolicy) -> MCPAppCSP +public static func headerValue(for csp: MCPAppCSP) -> String +public static func isSafeOrigin(_ origin: String) -> Bool +public static func revalidate(_ record: ToolCallRecord, source: any MCPAppWidgetSource, cache: MCPAppWidgetCache, +public static func revalidate(recordID: String, resourceURI: String, callDescriptor: MCPToolDescriptor?, +public static func session( +public static func toolInput(from arguments: GeneratedContent?) -> [String: MCPValue] +public static let closed = MCPAppsSandboxPolicy() +public static let mcpAppsExtensionKey = "io.modelcontextprotocol/ui" +public static let mimeType = "text/html;profile=mcp-app" +public static let protocolVersion = "2026-01-26" +public struct MCPAppCSP: Sendable, Equatable +public struct MCPAppPermissions: Sendable, Equatable +public struct MCPAppResource: Sendable, Equatable +public struct MCPAppResourceInfo: Sendable, Equatable +public struct MCPAppView: NSViewRepresentable +public struct MCPAppsAuditEvent: Sendable, Equatable +public struct MCPAppsBridgeConfiguration: Sendable +public struct MCPAppsDefaultPolicy: MCPAppsBridgePolicy +public struct MCPAppsHostActions +public struct MCPAppsSandboxPolicy: Sendable, Equatable +public struct MCPAppsSessionBackend: MCPAppsBackend +public struct MCPServerManagerBackend: MCPAppsBackend +public struct MCPServerManagerWidgetSource: MCPAppWidgetSource +public struct MCPToolInputField: Identifiable, Sendable, Equatable +public struct MCPToolInputForm: Sendable, Equatable +public struct Revalidation: Sendable +public var allowedConnectOrigins: [String] +public var allowedFrameOrigins: [String] +public var allowedResourceOrigins: [String] +public var allowedTools: Set? +public var allowsSameOrigin: Bool +public var availableDisplayModes: [String] +public var baseUriDomains: [String] = [] +public var camera = false +public var clipboardWrite = false +public var connectDomains: [String] = [] +public var csp: MCPAppCSP +public var decision: MCPAppRecreationDecision +public var description: String +public var domain: String? +public var effectiveCSP: MCPAppCSP +public var effectivePermissions: MCPAppPermissions +public var frameDomains: [String] = [] +public var geolocation = false +public var hasRequiredFields: Bool { fields.contains { $0.isRequired } } +public var hostContext: [String: MCPValue] +public var hostName: String +public var hostVersion: String +public var id: String { name } +public var isEmpty: Bool +public var isEmpty: Bool { !(camera || microphone || geolocation || clipboardWrite) } +public var isEmpty: Bool { fields.isEmpty } +public var isInspectable: Bool +public var limit: Int +public var maxBytes: Int +public var maxMessageBytes: Int +public var microphone = false +public var openLink: @MainActor (URL) -> Void +public var permissions: MCPAppPermissions +public var prefersBorder: Bool? +public var requestDisplayMode: @MainActor (_ mode: String) -> Bool +public var resource: MCPAppResource? +public var resourceDomains: [String] = [] +public var sendMessage: @MainActor (String) -> Void +public var storedBytes: Int { index.size.values.reduce(0, +) } +public var storedVersions: Set { Set(index.size.keys) } +public var teardownTimeout: Duration +public var updateModelContext: @MainActor (_ content: [MCPValue], _ structuredContent: MCPValue?) -> Void +public var widgetChanged: Bool +``` diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 8380466..b158fcd 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -889,6 +889,11 @@ signed in" behavior works. > (connect → build tools → run a turn); [`components-demo`](../examples/components-demo/) > additionally exercises resources and prompts. +An app that uses `LocalLMLab` doesn't make this: it uses `lab.mcp`, set up with +`LocalLMLab.Configuration(…, mcp: MCPSettings(…))` ([§6a](#6a-the-model-layer-local-models-routing-sessions)) — +`makeSession` takes MCP tools from `lab.mcp`, so a second manager would be invisible to it. The +calls below are the same on `lab.mcp`. + ```swift let manager = MCPServerManager() // NOT a singleton — you own the instance From bd2d7a5b12139647ea7b87b9c25c9e69dc3834bf Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 19:21:28 -0700 Subject: [PATCH 18/73] docs: bring the MCP examples' annotations up to date with their 2.0 source MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Checked every MCP-using example's annotated code against its current source (code only; markers, comments, joined lines and elisions ignored). - repo-qa-local, aiql: re-quoted from the 2.0 source — the server goes into lab.mcp and makeSession(includeMCPTools: true) offers its tools (aiql: setFileBackedOutput instead of wrapping with FileBackedTool.mcp); turns run on session.respond. Prose, tallies and size-table rows rewritten (repo-qa-local 96/129, aiql 402/459). - plate-today: the error path uses GenerationErrorDescription (tally 18). - code-buddy: Ctrl-C watch moved into startSigintWatch(_:), `default:` over SessionEvent, current git/test tool descriptions. Unchanged and already current: repo-qa, plate-today-tools, components-demo, workspace-buddy-local, security-demo, vistanova, mcp-chat (their remaining differences are deliberate line-joins, shortened prompts and excerpt ordering). Co-Authored-By: Claude --- docs/annotated-examples.md | 474 ++++++++++++++++++++++++------------- 1 file changed, 307 insertions(+), 167 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index 8954deb..b3087ea 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -29,7 +29,7 @@ non-comment lines; these examples are commented far more heavily than production |---|--:|--:|---| | [`repo-qa`](#examplesrepo-qa) | 66 | 115 | Apple's on-device model calling a real MCP server's tools, built from its live schema — no `Arguments` structs | | [`os-matrix`](#examplesos-matrix) | 74 | 97 | one binary that runs on macOS 26 **and** 27, model families gated by OS at registration | -| [`repo-qa-local`](#examplesrepo-qa-local) | 96 | 134 | `repo-qa` again, but the answer comes from a downloaded open-weight MLX model, **pinned** to a reviewed commit (the model layer) | +| [`repo-qa-local`](#examplesrepo-qa-local) | 96 | 129 | `repo-qa` again, but the answer comes from a downloaded open-weight MLX model, **pinned** to a reviewed commit (the model layer) | | [`components-demo`](#examplescomponents-demo) | 141 | 189 | a working "add / manage MCP servers" screen from prebuilt `Components` views, no MCP UI written | | [`plate-today-tools`](#examplesplate-today-tools) | 149 | 235 | Calendar + Reminders + Todoist (OAuth MCP) → a spoken-language day summary, on Core's ready-made tools | | [`workspace-buddy`](#examplesworkspace-buddy) | 173 | 227 | sandboxed AI edits to a user-picked folder, on-device model, a security-scoped bookmark that survives relaunch | @@ -38,7 +38,7 @@ non-comment lines; these examples are commented far more heavily than production | [`model-switch`](#examplesmodel-switch) | 328 | 415 | GPT / Claude online / OpenRouter + on-device, one chat call site, provider-run web search + citations (3 files) | | [`security-demo`](#examplessecurity-demo) | ~250 | ~490 | a "Security" panel → `limited(toMaxImpact:)` (which tools) + `ConfirmingToolAuthorizer` (whether they ask), a frontier model against Calendar + Todoist MCP (6 files) | | [`code-buddy`](#examplescode-buddy) | 315 | 413 | a CLI coding agent: two models with routing, workspace + host `Process` tools, MCP, a persistent REPL session (2 files) | -| [`aiql`](#examplesaiql) | 395 | 494 | a plain-English request → one read-only SQL `SELECT` over an MCP dataset → the CSV you asked for, sandboxed SwiftUI, zero fabricated values; a pinned default model and a trust policy for the free-text picker | +| [`aiql`](#examplesaiql) | 402 | 459 | a plain-English request → one read-only SQL `SELECT` over an MCP dataset → the CSV you asked for, sandboxed SwiftUI, zero fabricated values; a pinned default model and a trust policy for the free-text picker | | [`vistanova`](#examplesvistanova) | 931 | 1,294 | a tiny local search engine: web search through a Tavily MCP server on one local model, summaries from a **pinned** MLX model on another; defends against a model that skips the tool call (7 files) | | [`mcp-chat`](#examplesmcp-chat) | 938 | 1,026 | a chat with a local model where an MCP server's own interactive view (Todoist's task list) shows up under the tool call: the conversation is `hostTranscript`, the view is `MCPAppsHost` (4 files, views excerpted) | | [`components-updates-demo`](#examplescomponents-updates-demo) | 151 | 170 | the `Components` model **onboarding**, **update** and **versions** views, driven by simulated sources so every state is reachable | @@ -271,7 +271,7 @@ final class PlateTodayModel: ObservableObject { let response = try await session.respond(to: prompt) state = .ready(response.content) } catch { - state = .failed("\(error)") + state = .failed(await GenerationErrorDescription.describe(error)) // ← SDK } } } @@ -370,7 +370,7 @@ struct PlateTodayApp: App { } ``` -**Tally**: of ~230 lines of actual code (excluding comments/blank lines), 17 touch the SDK directly +**Tally**: of ~230 lines of actual code (excluding comments/blank lines), 18 touch the SDK directly (marked above) — everything else is ordinary SwiftUI state/view code and FoundationModels session setup that would look the same regardless of where the tools' data comes from. @@ -1107,6 +1107,21 @@ func parseArgs() -> Options { func note(_ s: String) { FileHandle.standardError.write(Data((s + "\n").utf8)) } +// SIGINT on a background dispatch queue, so the handler fires even while the main thread is +// parked in readLine(). Built in a non-isolated function on purpose: a closure formed in +// @MainActor scope is inferred main-actor-isolated, and Dispatch then traps when it runs off the +// main queue. +func startSigintWatch(_ interrupt: Interrupt) -> any DispatchSourceSignal { + signal(SIGINT, SIG_IGN) + let src = DispatchSource.makeSignalSource(signal: SIGINT, queue: .global()) + src.setEventHandler { + if interrupt.fire() { note("\nquitting…"); exit(130) } // process exit; no explicit session.cancel() here + note("\n^C interrupting this turn — Ctrl-C again to quit") + } + src.resume() + return src +} + // Ctrl-C policy: first press during a turn cancels *that turn* and returns to the prompt; // a press at an idle prompt — or a second press mid-turn — quits. final class Interrupt: @unchecked Sendable { @@ -1218,21 +1233,14 @@ func run() async { case .toolCallStarted(_, let name, _): if opts.verbose { note(" → \(name)") } case .toolCallFinished(_, let name, let failed, _): if opts.verbose { note(" \(failed ? "✗" : "✓") \(name)") } case .contextCompacted(let n): note(" (compacted \(n) transcript entries)") - @unknown default: break // non-frozen — see sdk-guide §9 + default: break // the other SessionEvent cases (load progress, …) } } } - // Ctrl-C: SIG_IGN + a DispatchSource on a background queue, so the handler runs even - // while the main thread is parked in readLine(). + // Ctrl-C: first press cancels the running turn, a press at an idle prompt quits. let interrupt = Interrupt() - signal(SIGINT, SIG_IGN) - let sigint = DispatchSource.makeSignalSource(signal: SIGINT, queue: .global()) - sigint.setEventHandler { - if interrupt.fire() { note("\nquitting…"); exit(130) } // process exit; no explicit session.cancel() here - note("\n^C interrupting this turn — Ctrl-C again to quit") - } - sigint.resume() + let sigint = startSigintWatch(interrupt) // One turn. streamResponse snapshots are *usually* append-only — but not across a tool // call, and not when a reasoning model drops its block once the answer begins. @@ -1356,12 +1364,12 @@ struct GitTool: Tool { "rev-parse", "describe", "remote", "tag", "shortlog", "grep", "cat-file", "reflog", ] @Generable struct Arguments { - @Guide(description: "A read-only git subcommand with its args, e.g. \"status\", \"log --oneline -10\".") + @Guide(description: "A read-only git subcommand with its args, e.g. \"status\", \"diff HEAD~1 -- Sources\", \"log --oneline -10\".") var command: String } let name = "git" var description: String { - "Runs a read-only git command in the workspace. Allowed: \(Self.readOnlySubcommands.sorted().joined(separator: ", ")). Mutating commands are refused — make edits with editWorkspaceFile instead." + "Runs a read-only git command in the workspace. Allowed: \(Self.readOnlySubcommands.sorted().joined(separator: ", ")). Mutating commands (commit, push, reset, checkout, clean, rebase) are refused — make edits with editWorkspaceFile instead." } func call(arguments: Arguments) async throws -> String { try Task.checkCancellation() // a tool call queued after a Ctrl-C never launches @@ -1380,12 +1388,12 @@ struct RunTestsTool: Tool { let root: URL let command: [String] // e.g. ["swift", "test"] or ["npm", "test", "--silent"] @Generable struct Arguments { - @Guide(description: "Optional substring to pass to the test runner's filter, to run a subset.") + @Guide(description: "Optional substring to pass to the test runner's filter, to run a subset. Omit to run all tests.") var filter: String? } let name = "run_tests" var description: String { - "Runs the project's test suite (`\(command.joined(separator: " "))`) in the workspace and returns the output." + "Runs the project's test suite (`\(command.joined(separator: " "))`) in the workspace and returns the output. Pass `filter` to run a subset." } func call(arguments: Arguments) async throws -> String { try Task.checkCancellation() @@ -1415,22 +1423,27 @@ make Ctrl-C in the REPL terminate a running `swift test` instead of orphaning it ## `examples/repo-qa-local` -*`Sources/RepoQALocal/main.swift` — 96 lines of code (134 with comments) — ~30 more than `repo-qa`, all of it the model layer.* - -The **minimal** model-layer example: [`repo-qa`](#examplesrepo-qa) above, -with the ~30 lines that swap Apple's on-device model for an open-weight MLX model you download and -run locally — including the **pin**: the default model is tied to the exact Hugging Face commit the example -was tried against (`pinnedRevisions`), and a model you pick with `--model` is pinned on first download -(`MLXFilePinStore`, trust on first use), so a later fresh download never silently fetches whatever `main` -has become. The Deepwiki / `MCPTool` half is a verbatim copy of `repo-qa`'s — diff the two to see -exactly what adopting the model layer costs. One of several examples linking -`LocalLMLabSDKInference` (`// ← SDK (Inference)`); `// ← SDK` is Core as elsewhere. +*`Sources/RepoQALocal/main.swift` — 96 lines of code (129 with comments).* + +The **minimal** model-layer example: [`repo-qa`](#examplesrepo-qa) above, with Apple's +on-device model swapped for an open-weight MLX model you download and run locally — including the +**pin**: the default model is tied to the exact Hugging Face commit the example was tried against +(`pinnedRevisions`), and a model you pick with `--model` is pinned on first download +(`MLXFilePinStore`, trust on first use), so a later fresh download never silently fetches whatever +`main` has become. The Deepwiki half is `repo-qa`'s, with one 2.0 difference: the server goes into +**`lab.mcp`** — the lab's one MCP manager — and `makeSession` offers its enabled tools +(`includeMCPTools`), instead of the example building `MCPTool`s itself. A new server's tools start +off, so the example turns on the ones it wants (`MCPTool(descriptor:manager:)` is used only to skip +a tool whose schema doesn't build). The turn runs on the session's own `respond(to:)`. One of +several examples linking `LocalLMLabSDKInference` (`// ← SDK (Inference)`); `// ← SDK` is Core as +elsewhere. ```swift // repo-qa-local — repo-qa, but the answer comes from an open-weight model you download and run -// locally (via MLX) instead of Apple's on-device model. The Deepwiki / MCPTool half is a -// verbatim copy of repo-qa's; the only difference is the ~20 lines that set up the 1.0 model -// layer and swap `LanguageModelSession(tools:)` for `lab.makeSession(route:tools:)`. +// locally (via MLX) instead of Apple's on-device model. The Deepwiki half follows repo-qa's, with +// one change: the MCP server goes into `lab.mcp` (the lab's manager — an app that uses LocalLMLab +// never makes an `MCPServerManager` of its own), and `lab.makeSession(route:)` builds the MCP tools +// from it, instead of repo-qa's own manager + `MCPTool`s + `LanguageModelSession(tools:)`. // // swift run RepoQALocal anthropics/claude-code "What is the plugin system?" // swift run RepoQALocal facebook/react # default question @@ -1442,8 +1455,8 @@ exactly what adopting the model layer costs. One of several examples linking import Foundation import FoundationModels -import LocalLMLabSDKCore // ← SDK -import LocalLMLabSDKInference // ← SDK (Inference) +import LocalLMLabSDKCore // ← SDK +import LocalLMLabSDKInference // ← SDK (Inference) func note(_ s: String) { FileHandle.standardError.write(Data((s + "\n").utf8)) } @@ -1479,14 +1492,14 @@ func run() async { // new version, then update the repo and its commit together. let shippedPins = ["mlx-community/Qwen3-8B-4bit": "545dc4251c05440727734bcd94334791f6ab0192"] let mlx = MLXModelProvider(residentModelLimit: 1, pinnedRevisions: shippedPins, pinStore: MLXFilePinStore()) // ← SDK (Inference) - let lab = LocalLMLab(configuration: .init(providers: [mlx, SystemModelProvider()])) // ← SDK - let modelID = useApple ? ModelID.system : ModelID(scheme: "mlx", rest: modelRepo)! // ← SDK + let lab = LocalLMLab(configuration: .init(providers: [mlx, SystemModelProvider()])) // ← SDK + let modelID = useApple ? ModelID.system : ModelID(scheme: "mlx", rest: modelRepo)! // ← SDK lab.models.route(.local, to: modelID) // ← SDK - note("model: \(modelID) · SDK \(LocalLMLabSDKVersion.current)") // ← SDK + note("model: \(modelID) · SDK \(LocalLMLabSDKVersion.current)") // ← SDK // Preflight + download the MLX weights on first run. (Nothing to download for `--apple`.) - if !useApple, case .notDownloaded = lab.models.availability(for: modelID) { // ← SDK - if let pre = try? await mlx.validate(modelRepo), !pre.passed { // ← SDK (Inference) + if !useApple, case .notDownloaded = lab.models.availability(for: modelID) { // ← SDK + if let pre = try? await mlx.validate(modelRepo), !pre.passed { // ← SDK (Inference) note("pre-flight failed (\(pre.failedStage?.rawValue ?? "?")): \(pre.detail ?? "")") exit(1) } @@ -1502,15 +1515,14 @@ func run() async { note("download failed: \(error)"); exit(1) } } - if !useApple, let pin = mlx.effectivePin(for: modelRepo) { // ← SDK (Inference) + if !useApple, let pin = mlx.effectivePin(for: modelRepo) { // ← SDK (Inference) note("pinned to \(pin.revision.prefix(7)) (\(pin.source == .shipped ? "shipped with this example" : "first download"))") } - // --- everything below is repo-qa, unchanged --- + // --- the Deepwiki half: repo-qa's, with the server in lab.mcp --- - let manager = MCPServerManager() // ← SDK note("Connecting to Deepwiki…") - let connectResult = await manager.addServer( // ← SDK + let connectResult = await lab.mcp.addServer( // ← SDK url: URL(string: "https://mcp.deepwiki.com/mcp")!, displayName: "Deepwiki" ) @@ -1518,40 +1530,46 @@ func run() async { note("Could not connect to Deepwiki: \(connectResult)"); return } - // Build a Tool for each of Deepwiki's tools from its own live schema. `read_wiki_contents` is - // skipped by name: it dumps a repo's entire wiki unscoped (~165K tokens for anthropics/ - // claude-code in one call), which `MCPTool` can't know from the schema — that curation is the - // app's job (see docs/sdk-guide.md §3). A tool whose schema doesn't build is skipped, not fatal. - var tools: [any Tool] = [] + // A new server's tools start disabled (a security default: nothing reaches the model until the + // host opts in), so enable the ones this example uses. `read_wiki_contents` stays off: it dumps + // a repo's entire wiki unscoped (~165K tokens for anthropics/claude-code in one call), which + // `MCPTool` can't know from the schema — that curation is the app's job (see + // docs/sdk-guide.md §3). A tool whose schema doesn't build is skipped, not fatal (makeSession + // would otherwise fail on it). + var enabled: [String] = [] for descriptor in state.tools { guard descriptor.name != "read_wiki_contents" else { note("Skipping \(descriptor.name): excluded by this example.") continue } - do { tools.append(try MCPTool(descriptor: descriptor, manager: manager)) } // ← SDK (Path A) - catch { note("Skipping \(descriptor.name): \(error)") } + do { _ = try MCPTool(descriptor: descriptor, manager: lab.mcp) } // ← SDK + catch { note("Skipping \(descriptor.name): \(error)"); continue } + lab.mcp.setToolEnabled(server: state.id, tool: descriptor.name, enabled: true) // ← SDK + enabled.append(descriptor.name) } - guard !tools.isEmpty else { note("Deepwiki didn't offer any usable tools."); return } - note("Built \(tools.count) tool(s) from Deepwiki's live schema: \(tools.map(\.name).joined(separator: ", "))") + guard !enabled.isEmpty else { note("Deepwiki didn't offer any usable tools."); return } + note("Enabled \(enabled.count) tool(s) from Deepwiki's live schema: \(enabled.joined(separator: ", "))") - // The one line that changes from repo-qa: lab.makeSession(route:tools:) instead of - // LanguageModelSession(tools:) — same FoundationModels session underneath, just backed by - // whichever model the route points at. + // Where repo-qa makes `LanguageModelSession(tools:)`, this calls lab.makeSession(route:) — same + // FoundationModels session underneath, backed by whichever model the route points at, with + // lab.mcp's enabled tools included (`includeMCPTools`, on by default). let instructions = "You answer questions about GitHub repositories using the documentation tools available to you. Always ground your answer in what the tools actually return — don't answer from general knowledge if a tool call would give a more specific, current answer." let session: LocalLMLabSession // ← SDK do { - session = try lab.makeSession(route: .local, tools: tools, instructions: instructions, includeMCPTools: false) // ← SDK + session = try lab.makeSession(route: .local, instructions: instructions, includeMCPTools: true) // ← SDK } catch { note("makeSession failed: \(error)"); exit(1) } let prompt = "Regarding the GitHub repository \"\(repoName)\": \(effectiveQuestion)" note("\nAsking: \(prompt)\n") + // The session's own turn method (repo-qa calls `session.respond(to:)` on its + // LanguageModelSession). Not `session.languageModelSession.respond`: that escape hatch skips + // the context-overflow retry and the per-turn MCP tool refresh. do { - let response = try await session.languageModelSession.respond(to: prompt) // ← SDK - print(response.content) + print(try await session.respond(to: prompt)) // ← SDK } catch { - note("Error: \(await GenerationErrorDescription.describe(error))") // ← SDK + note("Error: \(await GenerationErrorDescription.describe(error))") // ← SDK } session.cancel() // ← SDK } @@ -1563,14 +1581,14 @@ if #available(macOS 26.0, *) { } ``` -**Tally**: of the file's 96 non-comment/non-blank lines, 19 touch the SDK directly (marked above) -— and everything below the `--- everything below is repo-qa, unchanged ---` marker is -`repo-qa`'s code (its long comments condensed) except the one `lab.makeSession` line. The model layer itself is -~8 lines (`MLXModelProvider` / `LocalLMLab` / `ModelID` / `route` / `availability` / `validate` / -`download`, plus the diagnostic `note(...)` line that reads `LocalLMLabSDKVersion.current`), and pinning adds -three more: the `shippedPins` dictionary and `pinStore:` argument on `MLXModelProvider`, and an -`effectivePin(for:)` call that reports which version is in use and where the pin came from (`.shipped` vs `.captured`); -`--apple` proves the same route can point at Apple's on-device model with no other change. +**Tally**: of the file's 96 non-comment/non-blank lines, 19 touch the SDK directly (marked above). +The model layer is ~8 lines (`MLXModelProvider` / `LocalLMLab` / `ModelID` / `route` / +`availability` / `validate` / `download`, plus the diagnostic `note(...)` line that reads +`LocalLMLabSDKVersion.current`); pinning adds the `shippedPins` dictionary, the `pinStore:` argument +and an `effectivePin(for:)` call that reports which version is in use and where the pin came from +(`.shipped` vs `.captured`). The MCP half is `lab.mcp.addServer`, a `setToolEnabled` per tool, and +`makeSession(includeMCPTools: true)`; `--apple` proves the same route can point at Apple's on-device +model with no other change. ## `examples/workspace-buddy-local` @@ -2400,33 +2418,54 @@ the entire confirmation UI. Nothing in the three panel views touches the SDK — ## `examples/aiql` -*`Sources/AIQL/AIQLApp.swift` — 395 lines of code (494 with comments) — one file: view model + pipeline + SwiftUI UI. The -`FolderAccess` enum and the `ContentView` UI are elided below.* +*`Sources/AIQL/AIQLApp.swift` — 402 lines of code (459 with comments) — one file: view model + +pipeline + SwiftUI UI. The `FolderAccess` enum and the `ContentView` UI are elided below.* -The **`FileBackedTool` + `loadTable` + `sqlQuery`** showcase (`sdk-guide.md` +The **file-backed MCP tools + `loadTable` + `sqlQuery`** showcase (`sdk-guide.md` [§8b](sdk-guide.md#8b-filebackedtool--the-aiql-data-verbs-a-mechanical-mcp-dataset--csv-pipeline)): type a local model, an MCP data source, and a plain-English request; the app pulls the dataset into a file (so the raw payload never enters the model's context), `loadTable`s it into an ephemeral SQLite table, and has the model write **one read-only `SELECT`** to a CSV. The model names a table and describes one query — it never handles a row, and can't drop a step, because there's only ever one query call. `WHERE` / `BETWEEN` / `ORDER BY … LIMIT` / `JOIN` run inside -SQLite. Links `LocalLMLabSDKInference` for the MLX model. Because the model field is free text, the app can't pin -everything ahead of time, so it layers three supply-chain controls: the default model is **pinned** to a -reviewed commit (`pinnedRevisions:`), any other model is pinned on first download (`MLXFilePinStore`), and a -**trust policy** (`MlxCommunityOnly`) limits which repos can be fetched at all; downloads are hash-verified by -default. `FolderAccess` is verbatim from -`workspace-buddy-local` (`sdk-guide.md` §8) and elided. +SQLite. In 2.0 the data tools come from **`lab.mcp`**: the app adds the server there, turns on its +four most data-like tools, and sets `lab.mcp.setFileBackedOutput(MCPFileBackedOutput(root:…), server:)` +for the run's folder, so `makeSession(includeMCPTools: true)` offers each with a `saveAs` argument +that writes its result to a file. Links `LocalLMLabSDKInference` for the MLX model. Because the model +field is free text, the app can't pin everything ahead of time, so it layers three supply-chain +controls: the default model is **pinned** to a reviewed commit (`pinnedRevisions:`), any other model +is pinned on first download (`MLXFilePinStore`), and a **trust policy** (`MlxCommunityOnly`) limits +which repos can be fetched at all; downloads are hash-verified by default. `FolderAccess` is +verbatim from `workspace-buddy-local` (`sdk-guide.md` §8) and elided. ```swift +// AIQL — "ask your data". A SwiftUI app aimed at someone who lives in a marketing tool, not a +// terminal: type a local model, an MCP data source, and a plain-English request; press Go. +// +// Go → (1) connect to the MCP server (public, or an OAuth sign-in in the browser) +// (2) download the local model if it isn't cached yet +// (3) run the pipeline: pull the dataset into a file (FileBackedTool — the raw payload +// never enters the model's context), loadTable it into an ephemeral SQLite table, +// then one read-only sqlQuery (SELECT) → a CSV in the folder you chose. +// +// The model only writes the SQL — the host runs it read-only and the rows never enter the +// model's context, so there is nothing to fabricate. loadTable + sqlQuery are Core tools; see +// docs/sdk-guide.md §8b. +// +// Structure borrowed from: workspace-buddy-local (SwiftUI + App Sandbox + MLX model + folder +// picker + security-scoped bookmark), plate-today (MCP client + OAuth redirect via AppDelegate), +// repo-qa (tools from a live MCP schema — here built by makeSession from lab.mcp, file-backed). + import AppKit import Foundation import FoundationModels -import LocalLMLabSDKCore // ← SDK -import LocalLMLabSDKInference // ← SDK (Inference) +import LocalLMLabSDKCore // ← SDK +import LocalLMLabSDKInference // ← SDK (Inference) import SwiftUI -// MARK: - FolderAccess { pickFolder / resolveBookmarkedFolder / withFolderAccessAsync } -// — verbatim from workspace-buddy-local (docs/sdk-guide.md §8); elided here. +// MARK: - Folder picker + security-scoped bookmark (docs/sdk-guide.md §8) — verbatim from workspace-buddy-local + +// … FolderAccess — elided (see the section intro) // MARK: - Trust policy @@ -2435,15 +2474,18 @@ import SwiftUI /// uploads your users will run. struct MlxCommunityOnly: MLXModelTrustPolicy { // ← SDK (Inference) func evaluate(repoID: String) async -> MLXModelTrustDecision { // ← SDK (Inference) - repoID.hasPrefix("mlx-community/") ? .allow : .deny(reason: "only mlx-community models are allowed in this app") + repoID.hasPrefix("mlx-community/") ? .allow : .deny(reason: "only mlx-community models are allowed in this app") // ← SDK (Inference) } } +// MARK: - View model + @available(macOS 27.0, *) @MainActor final class AIQLModel: ObservableObject { enum Stage: Equatable { - case idle, connecting + case idle + case connecting case downloadingModel(Double) // 0…1 case running case done(fileName: String, csv: String, folder: URL) @@ -2458,24 +2500,51 @@ final class AIQLModel: ObservableObject { @Published var request = "" @Published private(set) var folderURL: URL? @Published private(set) var stage: Stage = .idle - @Published private(set) var steps: [String] = [] + @Published private(set) var steps: [String] = [] // friendly progress lines - private let manager = MCPServerManager() // ← SDK // The field is free text, so this app can't pin everything ahead of time. Two layers instead: // - the default model is pinned to a commit this app shipped with (developer-vouched); // - any other model is pinned on first download (`MLXFilePinStore`, kept outside the model // cache), so re-downloading later fetches the same version the user first got. // A trust policy limits which repos can be fetched at all; downloads are hash-verified by default. - private let mlx = MLXModelProvider( // ← SDK (Inference) + private let mlx = MLXModelProvider( // ← SDK (Inference) residentModelLimit: 1, - pinnedRevisions: [AIQLModel.defaultModelRepo: AIQLModel.defaultModelRevision], // ← SDK (Inference) - supplyChainPolicy: MLXSupplyChainPolicy(trustPolicy: MlxCommunityOnly()), // ← SDK (Inference) + pinnedRevisions: [AIQLModel.defaultModelRepo: AIQLModel.defaultModelRevision], + supplyChainPolicy: MLXSupplyChainPolicy(trustPolicy: MlxCommunityOnly()), // ← SDK (Inference) pinStore: MLXFilePinStore()) // ← SDK (Inference) - private lazy var lab = LocalLMLab(configuration: .init(providers: [mlx, SystemModelProvider()])) // ← SDK + // The MCP server goes into `lab.mcp`, the lab's manager — an app that uses LocalLMLab never + // makes an `MCPServerManager` of its own. + private lazy var lab = LocalLMLab(configuration: .init(providers: [mlx, SystemModelProvider()])) // ← SDK - init() { folderURL = FolderAccess.resolveBookmarkedFolder() } + init() { + folderURL = FolderAccess.resolveBookmarkedFolder() + } + + var isBusy: Bool { + switch stage { + case .connecting, .downloadingModel, .running: return true + default: return false + } + } + + var canGo: Bool { + folderURL != nil + && !modelRepo.trimmingCharacters(in: .whitespaces).isEmpty + && URL(string: serverURLString.trimmingCharacters(in: .whitespaces))?.scheme != nil + && !request.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty + && !isBusy + } + + func chooseFolder() { + if let url = FolderAccess.pickFolder() { folderURL = url } + } - // canGo / isBusy / chooseFolder / go() — plain view-model plumbing, elided. + func go() { + guard canGo else { return } + Task { await run() } + } + + private func step(_ line: String) { steps.append(line) } private func run() async { steps = [] @@ -2483,14 +2552,15 @@ final class AIQLModel: ObservableObject { guard let serverURL = URL(string: serverURLString.trimmingCharacters(in: .whitespaces)), serverURL.scheme != nil else { stage = .failed("That MCP server address doesn't look like a web link."); return } - guard let modelID = ModelID(scheme: "mlx", rest: modelRepo.trimmingCharacters(in: .whitespaces)) else { // ← SDK + guard let modelID = ModelID(scheme: "mlx", rest: modelRepo.trimmingCharacters(in: .whitespaces)) else { // ← SDK stage = .failed("The model name should look like mlx-community/Qwen3-8B-4bit."); return } lab.models.route(.local, to: modelID) // ← SDK // 1 — connect (triggers an OAuth browser sign-in automatically if the server needs one) stage = .connecting - let connection = await manager.addServer(url: serverURL, displayName: serverURL.host ?? "MCP server") // ← SDK + step("Connecting to \(serverURL.host ?? serverURL.absoluteString)…") + let connection = await lab.mcp.addServer(url: serverURL, displayName: serverURL.host ?? "MCP server") // ← SDK guard case .success(let server) = connection else { if case .failure(let error) = connection { stage = .failed("Couldn't connect to that MCP server — \(Self.describe(error))") @@ -2503,18 +2573,22 @@ final class AIQLModel: ObservableObject { stage = .failed("Connected, but that server didn't offer any tools to get data from."); return } step("Connected — \(server.tools.count) data tool(s) available.") + // Only this server's tools go to the model: one from an earlier Go stays connected in + // lab.mcp otherwise. (disconnect, not removeServer, which would forget its sign-in.) + for id in lab.mcp.servers.keys where id != server.id { lab.mcp.disconnect(id) } // ← SDK // 2 — download the model on first use - if case .notDownloaded = lab.models.availability(for: modelID) { // ← SDK + if case .notDownloaded = lab.models.availability(for: modelID) { // ← SDK stage = .downloadingModel(0) - if let preflight = try? await mlx.validate(modelRepo.trimmingCharacters(in: .whitespaces)), !preflight.passed { // ← SDK (Inference) + step("Downloading \(modelRepo) (first run only)…") + if let preflight = try? await mlx.validate(modelRepo.trimmingCharacters(in: .whitespaces)), !preflight.passed { // ← SDK (Inference) stage = .failed("That model didn't pass its check: \(preflight.detail ?? "unknown reason")."); return } do { - for try await event in mlx.download(modelRepo.trimmingCharacters(in: .whitespaces)) { // ← SDK (Inference) + for try await event in mlx.download(modelRepo.trimmingCharacters(in: .whitespaces)) { // ← SDK (Inference) if case .progress(_, _, let fraction) = event { stage = .downloadingModel(fraction) } } - if let pin = mlx.effectivePin(for: modelRepo.trimmingCharacters(in: .whitespaces)) { // ← SDK (Inference) + if let pin = mlx.effectivePin(for: modelRepo.trimmingCharacters(in: .whitespaces)) { // ← SDK (Inference) step("Pinned to version \(pin.revision.prefix(7)) (\(pin.source == .shipped ? "shipped with this app" : "the version you just downloaded")).") } } catch { @@ -2524,56 +2598,73 @@ final class AIQLModel: ObservableObject { // 3 — run the pipeline inside the security-scoped access window stage = .running + step("Working… this takes a few minutes.") let outcome = await FolderAccess.withFolderAccessAsync { root in - await self.runPipeline(prompt: prompt, root: root, serverTools: server.tools, serverID: server.id, modelID: modelID) + await self.runPipeline(prompt: prompt, root: root, serverTools: server.tools, serverID: server.id) } stage = outcome ?? .failed("Couldn't open the folder you chose — pick it again.") } - private func runPipeline(prompt: String, root: URL, serverTools: [MCPToolDescriptor], // ← SDK (type) - serverID: MCPServerID, modelID: ModelID) async -> Stage { // ← SDK (types) - // Wrap the server's tools as file-backed so the model can `saveAs` any of them. Cap the - // count — a small model degrades past ~8 tools; prefer names that look like "get a dataset". - let ranked = serverTools.sorted { Self.dataLikelihood($0.name) > Self.dataLikelihood($1.name) } - let dataTools: [any Tool] = ranked.prefix(4).compactMap { - try? FileBackedTool.mcp(descriptor: $0, manager: manager, root: root, inlineCharacterLimit: 8_000, // ← SDK — payload → file, receipt → model - followUp: "load it into a table with loadTable, then query it with one sqlQuery") - } - guard !dataTools.isEmpty else { return .failed("Couldn't read that server's tools — its data format isn't supported yet.") } - - // loadTable stages a JSON records file into an ephemeral SQLite table (auto-detects the - // records array, sniffs column types, explodes a nested array into a child table) and - // returns its CREATE TABLE. sqlQuery runs ONE read-only SELECT to a CSV. describeJson / - // csvInfo are for discovery + verification. The row data never reaches the model. - var tools: [any Tool] = dataTools - tools.append(LoadTableTool(root: root)) // ← SDK - tools.append(SQLQueryTool(root: root)) // ← SDK - tools.append(DescribeJSONTool(root: root)) // ← SDK - tools.append(CSVInfoTool(root: root)) // ← SDK - - let dataToolNames = dataTools.map(\.name).joined(separator: ", ") + private func runPipeline(prompt: String, root: URL, serverTools: [MCPToolDescriptor], serverID: MCPServerID) async -> Stage { // ← SDK + // Offer the server's data tools from lab.mcp. Cap the count — a small model degrades past + // ~8 tools; prefer names that look like "get a dataset" over admin/overview tools. A new + // server's tools start disabled, so enable the chosen ones (and only those). + let ranked = serverTools.sorted { lhs, rhs in Self.dataLikelihood(lhs.name) > Self.dataLikelihood(rhs.name) } + let chosen = ranked.prefix(4).map(\.name) + for tool in serverTools { + lab.mcp.setToolEnabled(server: serverID, tool: tool.name, enabled: chosen.contains(tool.name)) // ← SDK + } + // File-backed: each data tool gets a `saveAs` argument that writes its raw result into the + // chosen folder, so the dataset never enters the model's context. Only for this run — + // the folder's security-scoped access ends with it. + lab.mcp.setFileBackedOutput( // ← SDK + MCPFileBackedOutput(root: root, inlineCharacterLimit: 8_000, // ← SDK + followUp: "load it into a table with loadTable, then query it with one sqlQuery"), + server: serverID) + defer { lab.mcp.setFileBackedOutput(nil, server: serverID) } // ← SDK + + let tools: [any Tool] = [ + LoadTableTool(root: root), // ← SDK + SQLQueryTool(root: root), // ← SDK + DescribeJSONTool(root: root), // ← SDK + CSVInfoTool(root: root), // ← SDK + ] + + let dataToolNames = chosen.joined(separator: ", ") let instructions = """ You answer a data question by loading the pulled data into tables and running ONE SQL \ - query. You never write row data yourself … - 1. Pick the ONE data tool whose result answers the question — from: \(dataToolNames) — \ - and call it with `saveAs` set to "raw/data.json" (a second dataset → "raw/data2.json"; \ - `saveAsAppend: true` for each further page). - 2. loadTable jsonPath "raw/data.json", a short tableName, recordsAt "". Read the \ - CREATE TABLE it returns — use ONLY those column names. A nested array is a child \ - table "__"; join it with ".
_id =
.id". - 3. sqlQuery — ONE call: one SELECT, outputPath "out.csv". A numeric range → BETWEEN; \ - "top N" → ORDER BY … LIMIT N; "highest first" → ORDER BY DESC (never a rank \ - column). The moment it succeeds you are done. - 4. csvInfo path "out.csv", then reply in one sentence with the column names and row \ - count. Do not print the rows. - """ // (full prompt in the source) + query. You never write row data yourself. + + Run these steps in order, without asking for confirmation: + + 1. Pick the ONE data tool whose result answers the question — from: \(dataToolNames) — and \ + call it with its `saveAs` argument set to "raw/data.json". Never call a data tool \ + without `saveAs`; the result is large. If the question needs a second dataset, pull \ + that too, to "raw/data2.json". For a paged source call it once per page with the same \ + `saveAs` path plus `saveAsAppend: true` until you have every page. + 2. loadTable jsonPath "raw/data.json", a short tableName, recordsAt "" (let it find the \ + records). Do the same for any second file. Read each CREATE TABLE it returns — use \ + ONLY those column names. A nested array comes back as a child table \ + "
__"; join it with ".
_id =
.id". + 3. sqlQuery — ONE call: one SELECT, outputPath "out.csv". Standard SQLite. A numeric \ + range → BETWEEN. "top N" → ORDER BY … LIMIT N. For "highest / most first" sort by \ + itself DESC, never a rank column. Combine two tables → JOIN on the columns whose \ + sample values match. The moment it succeeds you are done — do NOT call sqlQuery again. + 4. csvInfo path "out.csv" — then reply in one sentence with the column names and the \ + row count. Do not print the rows. + + If a tool returns text starting with "Error:", read it, fix that one call, and retry it. + """ let session: LocalLMLabSession // ← SDK do { - // effort: .off — skip the model's pass. The Qwen3 family has the template - // toggle; the pipeline is mechanical, so the reasoning trace buys nothing but latency. - session = try lab.makeSession(route: .local, tools: tools, instructions: instructions, // ← SDK - includeMCPTools: false, options: SessionOptions(effort: .off)) // ← SDK + // effort: .off — skip the model's pass. The Qwen3 family has the + // template toggle; the pipeline is mechanical (pick a tool, name a table, write + // one SELECT) so the reasoning trace buys nothing but latency. + session = try lab.makeSession(route: .local, tools: tools, instructions: instructions, // ← SDK + includeMCPTools: true, options: SessionOptions(effort: .off)) // ← SDK + } catch LocalLMLabError.mcp { // ← SDK + return .failed("Couldn't read that server's tools — its data format isn't supported yet.") } catch { return .failed("Couldn't start the model: \(error.localizedDescription)") } @@ -2583,52 +2674,98 @@ final class AIQLModel: ObservableObject { let stepTask = Task { @MainActor in for await event in session.events { // ← SDK switch event { - case .toolCallStarted(_, let name, _): + case .toolCallStarted(_, let name, _): // ← SDK self.step(Self.friendlyStep(for: name)) - case .toolCallFinished(_, let name, let failed, _) where failed: + case .toolCallFinished(_, let name, let failed, _) where failed: // ← SDK self.step(" · \(Self.friendlyStep(for: name)) hit a snag — retrying") - default: break + default: + break } } } defer { stepTask.cancel() } + // The session's own turn method, not `languageModelSession.respond`: that escape hatch + // skips the context-overflow retry, the MCP tool refresh and the host transcript. do { - _ = try await session.languageModelSession.respond(to: "Question: \(prompt)\n\nBegin with step 1 now.") // ← SDK + try await session.respond(to: "Question: \(prompt)\n\nBegin with step 1 now.") // ← SDK } catch { - return .failed(await GenerationErrorDescription.describe(error)) // ← SDK + return .failed(await GenerationErrorDescription.describe(error)) // ← SDK } - // The result CSV. Instructions ask for "out.csv"; fall back to the newest .csv if a weak - // model left it in the last stage file. + // The result CSV. Instructions ask for "out.csv"; if a weak model wrote it somewhere + // else, fall back to the most recently written .csv. var name = "out.csv" - if case .failure = WorkspaceAccess.readFile(in: root, path: "out.csv"), // ← SDK - case .success(let entries) = WorkspaceAccess.listFiles(in: root, subpath: nil) { // ← SDK + if case .failure = WorkspaceAccess.readFile(in: root, path: "out.csv"), // ← SDK + case .success(let entries) = WorkspaceAccess.listFiles(in: root, subpath: nil) { // ← SDK if let newest = entries .filter({ !$0.isDirectory && $0.name.hasSuffix(".csv") }) .max(by: { ($0.modifiedDate ?? .distantPast) < ($1.modifiedDate ?? .distantPast) }) { name = newest.name } } - guard case .success(let csv) = WorkspaceAccess.readFile(in: root, path: name) else { // ← SDK - return .failed("The model finished but didn't write a spreadsheet. Try rephrasing, or a larger model.") + guard case .success(let csv) = WorkspaceAccess.readFile(in: root, path: name) else { // ← SDK + return .failed("The model finished but didn't write a spreadsheet. Try rephrasing the request, or a larger model.") } step("Saved \(name).") return .done(fileName: name, csv: csv, folder: root) } - // describe(_:) over MCPServerError, dataLikelihood / friendlyStep string helpers — elided. + // MARK: helpers + + static func describe(_ error: MCPServerError) -> String { // ← SDK + switch error { + case .unreachable: return "the server could not be reached." + case .malformedResponse: return "the server returned something unexpected." + case .protocolMismatch: return "the server speaks a different MCP version." + case .notConnected: return "not connected." + case .toolNotFound: return "a tool went missing." + case .serverError(let message): return message + case .authorizationRequired: return "it needs you to sign in." + case .credentialRejected: return "it rejected the saved sign-in." + case .httpError(let status): return "it returned HTTP \(status)." + case .oauthRegistrationNotSupported: return "it needs a manually configured OAuth client." + case .responseTooLarge: return "the server's response was too large to read." + @unknown default: return "\(error)" + } + } + + static func dataLikelihood(_ name: String) -> Int { + let lower = name.lowercased() + var score = 0 + for keyword in ["list", "search", "get_all", "contacts", "records", "rows", "dataset", "export", "find", "query", "items", "results", "countries", "people"] where lower.contains(keyword) { + score += 2 + } + for keyword in ["overview", "schema", "help", "meta", "config", "auth", "whoami", "status", "count"] where lower.contains(keyword) { + score -= 2 + } + return score + } + + static func friendlyStep(for toolName: String) -> String { + switch toolName { + case "loadTable": return "Reading the data…" + case "describeJson": return "Looking at how the data is organised…" + case "sqlQuery": return "Building the spreadsheet…" + case "csvInfo": return "Checking the result…" + default: return "Fetching the data…" + } + } } -// MARK: - ContentView — ordinary SwiftUI (four text fields, a Go button, a status area). Elided. +// MARK: - UI + +@available(macOS 27.0, *) +// … ContentView — elided (see the section intro) // Handle aiql://oauth/callback through the AppDelegate, not SwiftUI's .onOpenURL — WindowGroup -// treats an open-URL event as a request for a new window. Same fix plate-today / components-demo use. +// treats an open-URL event as a request for a new window (confirmed in plate-today: signing in +// brought back a second window). Same fix LocalLM Lab's own app uses. @available(macOS 27.0, *) private final class AppDelegate: NSObject, NSApplicationDelegate { func application(_ application: NSApplication, open urls: [URL]) { for url in urls where url.scheme == "aiql" { - MCPOAuthRedirectListener.shared.handleRedirect(url) // ← SDK + MCPOAuthRedirectListener.shared.handleRedirect(url) // ← SDK } } } @@ -2640,31 +2777,34 @@ struct AIQLApp: App { @StateObject private var model = AIQLModel() init() { + // Distinct from LocalLM Lab's "locallmlab" and plate-today's "platetoday" schemes so the + // callbacks don't collide if more than one is installed — matches Info.plist's + // CFBundleURLTypes. MCPOAuthFlow.redirectURI = "aiql://oauth/callback" // ← SDK } var body: some Scene { - WindowGroup { ContentView(model: model) } - .windowResizability(.contentMinSize) - .handlesExternalEvents(matching: []) + WindowGroup { + ContentView(model: model) + } + .windowResizability(.contentMinSize) + .handlesExternalEvents(matching: []) } } ``` -**Tally**: the model layer is the same ~7 lines as `repo-qa-local` (`MLXModelProvider` / `LocalLMLab` -/ `route` / `availability` / `validate` / `download` / `makeSession`), plus the supply-chain controls above: -an `MLXModelTrustPolicy` conformance (~5 lines), and the `pinnedRevisions:` / `supplyChainPolicy:` / `pinStore:` -arguments on `MLXModelProvider`, and an `effectivePin(for:)` call that tells the user which version they got. Everything new here is the -pipeline: `FileBackedTool.mcp(descriptor:manager:root:)` wraps each MCP data tool so its payload -lands in a file instead of the model's context, and four data-verb `Tool`s -(`LoadTableTool` / `SQLQueryTool` / `DescribeJSONTool` / `CSVInfoTool`, marked above) do the -mechanical work: `loadTable` stages a JSON records file into an ephemeral SQLite table, -`sqlQuery` runs exactly one read-only `SELECT` to a CSV, and `describeJson`/`csvInfo` are for -discovery and verification — the row data itself never passes through the model. The -`instructions` string doing the orchestration is the real work of this example — the SDK surface -it drives is the four data-verb `Tool`s plus up to four `FileBackedTool.mcp`-wrapped MCP tools -(`ranked.prefix(4)`, marked above), eight one-line `Tool` instantiations at most, plus -`WorkspaceAccess` to read the result back. +**Tally**: 42 lines carry a marker (32 Core, 10 Inference). The model layer is the same ~7 lines as +`repo-qa-local` (`MLXModelProvider` / `LocalLMLab` / `route` / `availability` / `validate` / +`download` / `makeSession`), plus the supply-chain controls above: an `MLXModelTrustPolicy` +conformance (~5 lines), the `pinnedRevisions:` / `supplyChainPolicy:` / `pinStore:` arguments on +`MLXModelProvider`, and an `effectivePin(for:)` call that tells the user which version they got. The +MCP side is `lab.mcp.addServer`, `setToolEnabled` for the chosen four, and one +`setFileBackedOutput` (cleared when the run ends) — the server's tools need no wrapping. Four +data-verb `Tool`s (`LoadTableTool` / `SQLQueryTool` / `DescribeJSONTool` / `CSVInfoTool`) do the +mechanical work: `loadTable` stages a JSON records file into an ephemeral SQLite table, `sqlQuery` +runs exactly one read-only `SELECT` to a CSV, and `describeJson`/`csvInfo` are for discovery and +verification — the row data itself never passes through the model. The `instructions` string +doing the orchestration is the real work of this example; `WorkspaceAccess` reads the result back. ## `examples/vistanova` From a718be0ea45e0d76fb97ec80872c7b95319d625a Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 19:31:28 -0700 Subject: [PATCH 19/73] examples: run turns with session.respond / streamResponse (2.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - code-buddy, workspace-buddy-local, mlx-control-room (synced from the SDK repo, fb3424c): stream with session.streamResponse, which yields the reply so far as a String, instead of languageModelSession — the escape hatch that skips the SDK's turn handling. - vistanova (this repo's copy): the plain-text search turn and the summary turn use session.respond. The guided-generation search turn (respond(to:generating:)) stays on languageModelSession — there is no SDK turn method for guided generation — with a comment saying so. - annotated-examples.md updated to match. Built against the published v2.0.0-dev binaries: code-buddy, workspace-buddy-local, mlx-control-room (swift build, clean) and vistanova (xcodebuild — its manifest only vends the binaries, so swift build never compiled the app). code-buddy run live on Qwen3-8B. Co-Authored-By: Claude --- docs/annotated-examples.md | 28 +++++++++++-------- .../code-buddy/Sources/CodeBuddy/main.swift | 8 +++--- .../MLXControlRoom/MLXControlRoomApp.swift | 8 +++--- .../Sources/VistaNova/AppModel.swift | 15 ++++++---- .../WorkspaceBuddyLocalApp.swift | 8 +++--- 5 files changed, 37 insertions(+), 30 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index b3087ea..cb3ec88 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -1242,14 +1242,15 @@ func run() async { let interrupt = Interrupt() let sigint = startSigintWatch(interrupt) - // One turn. streamResponse snapshots are *usually* append-only — but not across a tool + // One turn, through the session's own turn method (not `languageModelSession`, the escape + // hatch that skips the SDK's turn handling). session.streamResponse yields the reply so far — + // *usually* append-only, but not across a tool // call, and not when a reasoning model drops its block once the answer begins. // So diff against what we actually printed rather than slicing at a running offset. func ask(_ prompt: String) async { do { var shown = "" - for try await partial in session.languageModelSession.streamResponse(to: prompt) { // ← SDK - let content = partial.content + for try await content in session.streamResponse(to: prompt) { // ← SDK if content.isEmpty || content == shown { continue } if content.hasPrefix(shown) { print(content.dropFirst(shown.count), terminator: "") } else if shown.hasPrefix(content) { continue } // snapshot shrank; already shown @@ -1742,14 +1743,14 @@ final class WorkspaceBuddyLocalModel: ObservableObject { } defer { events.cancel() } - // Each snapshot is the whole answer so far. Usually append-only — but a reasoning + // session.streamResponse (the session's own turn method) yields the whole answer so far. Usually append-only — but a reasoning // model drops its block once the answer starts, and the snapshot can reset // across a tool call, so show the latest non-empty one rather than diffing. // (code-buddy does the careful append-only version — stdout can't un-print.) var text = "" - for try await snapshot in session.languageModelSession.streamResponse(to: request) { // ← SDK - guard !snapshot.content.isEmpty else { continue } - text = snapshot.content + for try await latest in session.streamResponse(to: request) { // ← SDK + guard !latest.isEmpty else { continue } + text = latest self.activity = nil self.state = .working(text) } @@ -3014,6 +3015,9 @@ final class AppModel { if supportsGuidedGeneration { do { + // Guided generation (`generating:`) has no SDK turn method, so this one call stays on + // Apple's session — the escape hatch — which also keeps the recovery below reading + // Apple's own error. let response = try await session.languageModelSession.respond( // ← SDK — a plain FoundationModels session to: query, generating: SearchResults.self) guard toolWasCalled else { throw SearchParseError.toolNotCalled } @@ -3026,7 +3030,7 @@ final class AppModel { } } else { // No guided generation on this model: plain text, parsed leniently for (title, url). - let response = try await session.languageModelSession.respond(to: query) // ← SDK + let reply = try await session.respond(to: query) // ← SDK — the session's own turn method // … } } @@ -3430,10 +3434,10 @@ final class ControlRoomModel: ObservableObject { let start = Date() var firstChunkAt: Date? var wordCount = 0 - for try await partial in session.languageModelSession.streamResponse(to: prompt) { // ← SDK - if firstChunkAt == nil, !partial.content.isEmpty { firstChunkAt = Date() } - lastOutput = partial.content - wordCount = partial.content.split(separator: " ").count + for try await partial in session.streamResponse(to: prompt) { // ← SDK + if firstChunkAt == nil, !partial.isEmpty { firstChunkAt = Date() } + lastOutput = partial + wordCount = partial.split(separator: " ").count } let elapsed = Date().timeIntervalSince(start) tokensPerSecond = elapsed > 0 ? Double(wordCount) / elapsed : nil diff --git a/examples/code-buddy/Sources/CodeBuddy/main.swift b/examples/code-buddy/Sources/CodeBuddy/main.swift index 4565c54..1314531 100644 --- a/examples/code-buddy/Sources/CodeBuddy/main.swift +++ b/examples/code-buddy/Sources/CodeBuddy/main.swift @@ -220,16 +220,16 @@ func run() async { let interrupt = Interrupt() let sigint = startSigintWatch(interrupt) - // One turn: stream the model's answer to stdout. streamResponse yields snapshots that - // are *usually* append-only — but not across a tool call, and not when a reasoning + // One turn: stream the model's answer to stdout, through the session's own turn method + // (not `languageModelSession`, the escape hatch that skips the SDK's turn handling). + // session.streamResponse yields the reply so far — *usually* append-only — but not across a tool call, and not when a reasoning // model drops its block once the answer proper begins. So diff against what we // actually printed: extend it when the snapshot grows, and when a snapshot diverges // (new segment) print it whole rather than slicing off its head. func ask(_ prompt: String) async { do { var shown = "" - for try await partial in session.languageModelSession.streamResponse(to: prompt) { - let content = partial.content + for try await content in session.streamResponse(to: prompt) { if content.isEmpty || content == shown { continue } if content.hasPrefix(shown) { print(content.dropFirst(shown.count), terminator: "") diff --git a/examples/mlx-control-room/Sources/MLXControlRoom/MLXControlRoomApp.swift b/examples/mlx-control-room/Sources/MLXControlRoom/MLXControlRoomApp.swift index c1584c0..04806ca 100644 --- a/examples/mlx-control-room/Sources/MLXControlRoom/MLXControlRoomApp.swift +++ b/examples/mlx-control-room/Sources/MLXControlRoom/MLXControlRoomApp.swift @@ -233,10 +233,10 @@ final class ControlRoomModel: ObservableObject { let start = Date() var firstChunkAt: Date? var wordCount = 0 - for try await partial in session.languageModelSession.streamResponse(to: prompt) { - if firstChunkAt == nil, !partial.content.isEmpty { firstChunkAt = Date() } - lastOutput = partial.content - wordCount = partial.content.split(separator: " ").count + for try await partial in session.streamResponse(to: prompt) { + if firstChunkAt == nil, !partial.isEmpty { firstChunkAt = Date() } + lastOutput = partial + wordCount = partial.split(separator: " ").count } let elapsed = Date().timeIntervalSince(start) tokensPerSecond = elapsed > 0 ? Double(wordCount) / elapsed : nil diff --git a/examples/vistanova/Sources/VistaNova/AppModel.swift b/examples/vistanova/Sources/VistaNova/AppModel.swift index 4c41a29..d0d054e 100644 --- a/examples/vistanova/Sources/VistaNova/AppModel.swift +++ b/examples/vistanova/Sources/VistaNova/AppModel.swift @@ -368,6 +368,9 @@ final class AppModel { if supportsGuidedGeneration { do { + // Guided generation (`generating:`) has no SDK turn method, so this one call stays on + // Apple's session (`languageModelSession`, the escape hatch) — which also keeps the + // guardrail-recovery below reading Apple's own error. let response = try await session.languageModelSession.respond(to: query, generating: SearchResults.self) guard toolWasCalled else { throw SearchParseError.toolNotCalled } return response.content.pages.map { SearchResultLink(title: $0.title, url: $0.url, snippet: $0.snippet) } @@ -384,12 +387,12 @@ final class AppModel { } } else { // No structured-output support on this model (see searchCapability(_:)) — a plain - // response.content String, parsed leniently for (title, url) pairs rather than + // String reply (the session's own turn method), parsed leniently for (title, url) pairs rather than // relying on the model to hit an exact format every time. - let response = try await session.languageModelSession.respond(to: query) - let links = Self.parsePlainTextLinks(response.content) + let reply = try await session.respond(to: query) + let links = Self.parsePlainTextLinks(reply) guard toolWasCalled, !links.isEmpty else { - throw toolWasCalled ? SearchParseError.noResultsParsed(rawText: response.content) : SearchParseError.toolNotCalled + throw toolWasCalled ? SearchParseError.noResultsParsed(rawText: reply) : SearchParseError.toolNotCalled } return links } @@ -509,9 +512,9 @@ final class AppModel { route: "summary", instructions: "Summarize the given web search results in 2-3 sentences, as one plain paragraph. No headers, no list, no commentary about the sources themselves.", includeMCPTools: false) - let response = try await session.languageModelSession.respond( + let reply = try await session.respond( to: "Search results for \"\(turn.searchQuery ?? turn.query)\":\n\(sources)") - threads[threadIdx].turns[turnIdx].summary = Self.stripThinkingBlock(response.content) + threads[threadIdx].turns[turnIdx].summary = Self.stripThinkingBlock(reply) HistoryStore.save(threads) } catch { lastError = await GenerationErrorDescription.describe(error) diff --git a/examples/workspace-buddy-local/Sources/WorkspaceBuddyLocal/WorkspaceBuddyLocalApp.swift b/examples/workspace-buddy-local/Sources/WorkspaceBuddyLocal/WorkspaceBuddyLocalApp.swift index 206ecc1..22224ac 100644 --- a/examples/workspace-buddy-local/Sources/WorkspaceBuddyLocal/WorkspaceBuddyLocalApp.swift +++ b/examples/workspace-buddy-local/Sources/WorkspaceBuddyLocal/WorkspaceBuddyLocalApp.swift @@ -199,15 +199,15 @@ final class WorkspaceBuddyLocalModel: ObservableObject { } defer { events.cancel() } - // Each snapshot is the whole answer so far. It's *usually* append-only, but a + // session.streamResponse (the session's own turn method) yields the whole answer so far. It's *usually* append-only, but a // reasoning model drops its block once the real answer starts, and the // snapshot can reset across a tool call — so just show the latest non-empty one // rather than diffing. (code-buddy does the careful append-only version, because // stdout can't un-print.) var text = "" - for try await snapshot in session.languageModelSession.streamResponse(to: request) { - guard !snapshot.content.isEmpty else { continue } - text = snapshot.content + for try await latest in session.streamResponse(to: request) { + guard !latest.isEmpty else { continue } + text = latest self.activity = nil self.state = .working(text) } From 7b639c3b0f779a3ffb10220bfc363721804eb657 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 19:38:01 -0700 Subject: [PATCH 20/73] docs: explain respond/streamResponse vs the languageModelSession escape hatch MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - sdk-guide.md §6a: new subsection "Running a turn, and when to use languageModelSession": a table of what a turn gets each way (tool approval / records / events work both ways; message at turn start, replyInProgress, turnContext, widget context, MCP tool refresh, overflow retry, on-device sampling, citations, cancel() only via respond/streamResponse), when the escape hatch is right (guided generation, reading the raw transcript, other Apple APIs), and what changes coming from Apple's methods (String results, error wrapping). - A short mention with a link where each affected feature is covered: makeSession (tool refresh + run-every-turn paragraph), sampling, "Building a chat app" (transcript, turnContext), events (.contextCompacted / .serverToolCall), RetryPolicy (1.x "wrappers" wording replaced), web-search citations, §3f widget context, capabilityProbe (.guidedGeneration), §12 languageModelSession. - migrating-to-2.0.md links the table. Co-Authored-By: Claude --- docs/migrating-to-2.0.md | 2 +- docs/sdk-guide.md | 89 ++++++++++++++++++++++++++++++++++------ 2 files changed, 77 insertions(+), 14 deletions(-) diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md index 505d90a..9f03366 100644 --- a/docs/migrating-to-2.0.md +++ b/docs/migrating-to-2.0.md @@ -105,7 +105,7 @@ passed `restoring:` before `tools:`). | What | 2.0 | |---|---| | Make a session | One `makeSession(route:tools:instructions:restoring:includeMCPTools:mcpAppHints:options:authorizer:)`. Continue a saved conversation with `restoring:` (instead of `instructions:` — passing both traps). | -| Run a turn | `session.respond(to:options:fromAppInstance:)` / `streamResponse(to:options:fromAppInstance:)`. `languageModelSession` is the escape hatch: a turn run on it directly skips the chat history, `turnContext`, widget context, the MCP tool refresh and `retryOnContextOverflow`. | +| Run a turn | `session.respond(to:options:fromAppInstance:)` / `streamResponse(to:options:fromAppInstance:)`. `languageModelSession` is the escape hatch: a turn run on it directly skips the chat history, `turnContext`, widget context, the MCP tool refresh and `retryOnContextOverflow` — table and when to still use it in [`sdk-guide.md` "Running a turn"](sdk-guide.md#running-a-turn-and-when-to-use-languagemodelsession). | | Set up MCP | `lab.mcp`, configured with `LocalLMLab.Configuration(providers:state:mcp:)` and `MCPSettings` (handlers, response limits, version negotiation, live updates). An app that uses `LocalLMLab` doesn't make its own `MCPServerManager`; making one directly is for apps without `LocalLMLab`. | | Describe a pending call | One `PendingToolCall` / `PendingToolCallSummary` initializer each; `initiator:` defaults to `.model`, `serverApproval:` to `nil`. | diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index b158fcd..b6a3833 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -696,7 +696,8 @@ through your session's authorizer as `ToolCallInitiator.app(instance:)`, recorde `hostTranscript` like the model's (tools the server marks app-only, such as a dashboard's polling, are counted per view instead of recorded). Sending a message as the user (`ui/message`), sharing context with the model (`ui/update-model-context`) and opening links are denied unless your -policy allows them. `ConfirmingToolAuthorizer` remembers the user's answer per view and tool, so a +policy allows them. Shared context reaches the model with the next `respond` / `streamResponse` +turn — a turn run on `languageModelSession` doesn't send it ([Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)). `ConfirmingToolAuthorizer` remembers the user's answer per view and tool, so a polling view asks once (destructive calls ask each time). ## 4. Keychain storage — automatic isolation, native API, sandbox-safe @@ -1143,6 +1144,8 @@ mlx.cancelDownload("mlx-community/Qwen3-8B-4bit") // Post-download smoke test — a real prompt + a trivial tool call. Authoritative for // that model's ModelCapabilities (some downloaded models can't reliably tool-call). let report = await mlx.capabilityProbe(installed.id) // ModelCapabilityReport +// report.capabilities: .toolCalling, .guidedGeneration (respond(to:generating:) — run on +// languageModelSession, see "Running a turn"), .reasoningToggle // This is authoritative. For a starting shortlist of what tool-calls and what doesn't, // see docs/tested-models.md — but always confirm your own model with capabilityProbe. @@ -1382,16 +1385,18 @@ function, `makeSession(route:tools:instructions:restoring:includeMCPTools:mcpApp - **MCP tools follow `lab.mcp`.** With `includeMCPTools` (the default) the session offers the enabled MCP tools and keeps following `lab.mcp`: turn a tool on or off, add a server, or change a server's trust, and the model has it on its next `respond` / `streamResponse` — same session. + (A turn run on `languageModelSession` directly doesn't pick changes up — [Running a turn](#running-a-turn-and-when-to-use-languagemodelsession).) For a fixed, per-session set of MCP tools, pass them in `tools:` with `includeMCPTools: false`. - **`restoring:`** continues a saved conversation (see [Building a chat app](#building-a-chat-app--hosttranscript-streamresponse-turncontext)); leave `instructions` nil then — the saved transcript carries its own (passing both traps). - **`mcpAppHints: true`** adds one line to the description of each MCP tool that has an MCP App view, so a model can prefer it when the user asks to *see* something ([§3f](#3f-mcp-apps-showing-a-servers-interactive-views)). -**Run turns with `respond` / `streamResponse`.** They keep `hostTranscript` up to date, send -`turnContext` and any widget context, refresh MCP tools and apply `retryOnContextOverflow`. -`languageModelSession` is Apple's session underneath — the escape hatch for what the SDK doesn't -surface (the raw `transcript` to save, say); a turn run on it directly skips all of the above. +**Run every turn with `respond` / `streamResponse`.** They are what make the rest of this +section work — the MCP tool refresh above, the sampling options below, `turnContext`, the chat +history, the overflow retry. `languageModelSession` is Apple's session underneath, and a turn run +on it directly skips all of that; what it costs, and when it's still the right call (guided +generation, reading the raw transcript), is in [Running a turn](#running-a-turn-and-when-to-use-languagemodelsession). **`options: SessionOptions`** carries per-call knobs — provider-native web search, sampling (`temperature` / `topP` / `topK` / `seed` / `maxOutputTokens`), and `effort`. `effort: .off` asks the model **not @@ -1435,8 +1440,61 @@ process launch occasionally differed, and the model itself can change with an OS seed → usually the same text" as the contract; don't build tests or caching on exact matches. (On MLX, where the weights are pinned, a seed is a stronger tool for replay and golden-output tests.) -If you call `session.languageModelSession.respond(...)` directly to opt out of the SDK's wrapper, -these options are **not** applied — pass `try options.appleGenerationOptions()` yourself. +These options are applied by `respond` / `streamResponse`. A turn run on +`session.languageModelSession` directly does **not** get them — pass +`try options.appleGenerationOptions()` yourself ([Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)). + +### Running a turn, and when to use `languageModelSession` + +> **Use `respond` / `streamResponse` for every turn.** Reach for `languageModelSession` only for +> what the SDK has no method for — guided generation, or reading the raw transcript. +> +> **Examples that use it:** every example built on `LocalLMLab` runs its turns this way; +> [`vistanova`](../examples/vistanova/) also shows the one escape-hatch case, guided generation. +> (The small examples that don't use `LocalLMLab` — `repo-qa`, `plate-today`, `workspace-buddy` — +> make Apple's `LanguageModelSession` themselves; with no SDK session there is nothing to skip.) + +```swift +let answer = try await session.respond(to: prompt) // the whole reply, a String +for try await soFar in session.streamResponse(to: prompt) { // the reply so far, a String + show(soFar) +} +``` + +`session.languageModelSession` is Apple's `LanguageModelSession` underneath, and you can call +`respond` / `streamResponse` on it directly. The tools still run through the SDK — so tool +approval, tool-call records and tool events work either way — but everything the SDK does *around* +a turn doesn't happen: + +| | `session.respond` / `streamResponse` | `session.languageModelSession.respond` / `.streamResponse` | +|---|---|---| +| Tool approval (`authorizer:`), tool-call records in `hostTranscript`, `.toolCallStarted` / `.toolCallFinished` | yes | yes — these live in the tools themselves | +| The user's message in `hostTranscript` as the turn starts; reply and reasoning when it ends; `replyInProgress` while streaming | yes | no message at turn start, no `replyInProgress`; text only after you call `reconcileServerToolArtifacts()` | +| `turnContext` (e.g. the date) and context an MCP App view shared, sent with the prompt | yes | no — the model never sees them | +| MCP tools turned on, or servers added, since the last turn | offered | not offered — the model session keeps the tools it had | +| `retryOnContextOverflow` | applied (`.contextCompacted`) | no — the overflow error reaches you | +| `SessionOptions` sampling on Apple's on-device model | applied | no — pass `try options.appleGenerationOptions()` yourself | +| `session.citations`, `.serverToolCall` (provider web search) | updated | only after `reconcileServerToolArtifacts()` | +| `session.cancel()` | stops the turn | doesn't — cancel your own `Task` | + +**When the escape hatch is the right call:** + +- **Guided generation** — `respond(to:generating:)` with a `@Generable` result type. The SDK has no + turn method for it, so call it on `languageModelSession`, then `session.reconcileServerToolArtifacts()` + so `hostTranscript` and `citations` catch up. For a downloaded model, check + `ModelCapabilities.guidedGeneration` (`capabilityProbe`) first. [`vistanova`](../examples/vistanova/)'s + structured search does exactly this, and uses `respond` for its plain-text fallback. +- **Reading the raw transcript** — `languageModelSession.transcript`, to save the model's memory + ([Building a chat app](#building-a-chat-app--hosttranscript-streamresponse-turncontext)) or to see + what it was given. Reading is fine; it's *running a turn* there that skips the SDK. +- **Any other Apple session API** the SDK doesn't wrap. + +**Coming from Apple's methods:** `respond` returns the reply as a `String` (Apple: a `Response`, read +`.content`); `streamResponse` yields `String`s, the reply so far — *usually* append-only, not across a +tool call or when a reasoning model's `` block gives way to the answer (Apple: snapshots, +`.content`). Failures arrive as `LocalLMLabError.generation(underlying:)`, cancellation as +`CancellationError`; `GenerationErrorDescription.describe(_:)` reads either. Per-call options are +`respond(to:options:)` — on-device model only, as above. ### Building a chat app — `hostTranscript`, `streamResponse`, `turnContext` @@ -1473,12 +1531,12 @@ if let reply = session.hostTranscript.replyInProgress { StreamingReply(reply) } when it starts and changes in place as it finishes; the reply and reasoning land when the turn ends (`replyInProgress` until then — cleared in the same update, so a view never shows both). A turn run on `languageModelSession` directly gets text only after - `reconcileServerToolArtifacts()`. + `reconcileServerToolArtifacts()` ([Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)). - **Tool calls are full records** (`ToolCallRecord`): state, arguments, the output the model saw, the full `MCPToolResult` (`mcpResult` — kept under a memory budget, spilled to disk past it), the tool's MCP App link (`app`), who made the call (`initiator`: the model, an MCP App view, or your app), and timestamps. -- **A model has no clock.** `turnContext` is called as each turn starts and its text goes ahead of +- **A model has no clock.** `turnContext` is called as each `respond` / `streamResponse` turn starts and its text goes ahead of the user's message — current even in a restored conversation, whose instructions were written long ago. (The ready-made `ClockTool` lets the model ask too, but a small model may not.) - **Tools of a newly added MCP server start off.** Show the user that, or the model gets no tools @@ -1531,6 +1589,9 @@ it), `.contextCompacted` (the retry-and-trim above), `.modelLoadProgress` (a loc load), `.routeSwitched` (the route this session was on just repointed to a different model — rare, but real if your app calls `lab.models.route(_:to:)` mid-session), and `.serverToolCall` (a provider running web search/fetch on its own infrastructure — §6b). +`.toolCallStarted` / `.toolCallFinished` arrive however you run the turn; `.contextCompacted` only +from `respond` / `streamResponse` (it's their retry), and `.serverToolCall` once +`reconcileServerToolArtifacts()` has run — which they do for you ([Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)). ### `ContextBudget` + `RetryPolicy` — surviving a long session @@ -1545,8 +1606,9 @@ rare, but real if your app calls `lab.models.route(_:to:)` mid-session), and - `session.retryOnContextOverflow = RetryPolicy(maxRetries: 2, compact: { transcript in myTrim(transcript) })` — when a turn throws `contextSizeExceeded`, the SDK calls your `compact` hook, rebuilds the session with the smaller transcript, emits `.contextCompacted`, - and retries. Only applies to the SDK's `respond` wrappers; with no `compact` hook it just - rethrows. + and retries. Applies to turns run with `respond` / `streamResponse`; a turn run on + `languageModelSession` directly isn't retried ([Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)). With no `compact` hook it + just rethrows. ### `ModelAvailability` — gray out a model and say why @@ -1651,7 +1713,8 @@ let answer = try await lab.makeSession(route: "chat").respond(to: prompt) `.serverToolCall(ServerToolActivity)` — its `.kind` is `.webSearch(queries:hits:)`, `hits` being `[WebHit]` (`url`/`title`/`snippet`) — so a "Searched: …" activity line doesn't need to wait for the final answer. `session.citations: [Citation]` (`url`/`title`/`citedText`) carries - the sources once the turn finishes — render these as footnotes, since a provider's web-search + the sources once the turn finishes (filled in by `respond` / `streamResponse`; after a turn on + `languageModelSession`, call `reconcileServerToolArtifacts()` first — [Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)) — render these as footnotes, since a provider's web-search answer without visible sources reads as an unverifiable claim. `SessionOptions.userLocation` (`city`/`region`/`country`/`timezone`) localizes results when the provider supports it — set it if your search-heavy queries are location-sensitive ("restaurants near me"). @@ -2870,7 +2933,7 @@ struct LocalLMLabSession: Sendable { var retryOnContextOverflow: RetryPolicy // .disabled by default; set before respond() var turnContext: (@Sendable () -> String?)? // sent ahead of every prompt (e.g. the date); not shown in hostTranscript var hostTranscript: HostTranscript { get } // the conversation as a UI shows it (below) - var languageModelSession: LanguageModelSession { get } // Apple's session — the escape hatch; a turn run on it skips the SDK's turn handling + var languageModelSession: LanguageModelSession { get } // Apple's session — the escape hatch; a turn run on it skips the SDK's turn handling ([Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)) var events: AsyncStream { get } var citations: [Citation] { get } // web-search sources accumulated this session ([§6b](#6b-online-providers--gpt-claude-online-openrouter-locallmlabsdkremote)) func reconcileServerToolArtifacts() // re-derive citations/events from the transcript after a manual edit From a8e47d1306ad0146c1417d43fad77506c4785a57 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 2 Oct 2026 19:51:17 -0700 Subject: [PATCH 21/73] Cascade republished 2.0.0-dev SDK checksums (Core, Inference, Remote, Claude) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All four 2.0.0-dev xcframeworks were rebuilt and re-uploaded on 2026-10-03 02:42–02:47 UTC (Core now carries the MCP MRTR Cancel/Decline fix and the MCP.mrtr log category). The pins from 8d5b7d7 no longer match the assets, so a fresh SwiftPM resolve would fail on checksum. Values are the release assets' SHA-256 digests. --- Components/Package.swift | 2 +- examples/README.md | 6 +++--- examples/aiql/Package.swift | 4 ++-- examples/code-buddy/Package.swift | 4 ++-- examples/mcp-chat/Package.swift | 2 +- examples/mlx-control-room/Package.swift | 4 ++-- examples/model-switch/Package.swift | 2 +- examples/os-matrix/Package.swift | 4 ++-- examples/plate-today-tools/Package.swift | 2 +- examples/plate-today/Package.swift | 2 +- examples/repo-qa-local/Package.swift | 4 ++-- examples/repo-qa/Package.swift | 2 +- examples/security-demo/Package.swift | 2 +- examples/vistanova/Package.swift | 4 ++-- examples/workspace-buddy-local/Package.swift | 4 ++-- examples/workspace-buddy/Package.swift | 2 +- 16 files changed, 25 insertions(+), 25 deletions(-) diff --git a/Components/Package.swift b/Components/Package.swift index bbf0519..09ff1f5 100644 --- a/Components/Package.swift +++ b/Components/Package.swift @@ -33,7 +33,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" ), ] diff --git a/examples/README.md b/examples/README.md index efa6316..2ea6794 100644 --- a/examples/README.md +++ b/examples/README.md @@ -173,7 +173,7 @@ let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease(url: "…/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2…"), "2.0.0-dev": SDKRelease(url: "…/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "ae20766f…"), + checksum: "20ac14db…"), ] ``` @@ -185,9 +185,9 @@ distinct `.xcframework.zip.sha256`; don't reuse Core's for another module): ```bash curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip.sha256 -# → ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9 +# → 20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKClaude-2.0.0-dev.xcframework.zip.sha256 -# → 5986c09b3e7c21b0b5ded462f35ffffd664f54609fda88310b8cfaab111dbf88 +# → b19b8a35eb1081d4aae3d8a1f12aae6e8a93f11f72016ddca6b2c44aa06ff2f9 ``` Then point `defaultSDKVersion` at it (works everywhere, Xcode included) or pass diff --git a/examples/aiql/Package.swift b/examples/aiql/Package.swift index 9e48e96..a1f14a4 100644 --- a/examples/aiql/Package.swift +++ b/examples/aiql/Package.swift @@ -42,9 +42,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/code-buddy/Package.swift b/examples/code-buddy/Package.swift index 024d117..82117f6 100644 --- a/examples/code-buddy/Package.swift +++ b/examples/code-buddy/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift index 7d34bf6..84f8208 100644 --- a/examples/mcp-chat/Package.swift +++ b/examples/mcp-chat/Package.swift @@ -36,7 +36,7 @@ let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-dev": SDKRelease( inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/mlx-control-room/Package.swift b/examples/mlx-control-room/Package.swift index aace797..1ef8875 100644 --- a/examples/mlx-control-room/Package.swift +++ b/examples/mlx-control-room/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/model-switch/Package.swift b/examples/model-switch/Package.swift index eda39f0..dea14eb 100644 --- a/examples/model-switch/Package.swift +++ b/examples/model-switch/Package.swift @@ -37,7 +37,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "88f8145fbaff6ff9d9ec7332abb8837ae4010b2fbbe390527e18ac50fdc77ad3" + remoteChecksum: "013873523fcbb03aa1ed03e808fc90d440cd7dc1ca8ffcbc326a19cfe81d2337" ), ] diff --git a/examples/os-matrix/Package.swift b/examples/os-matrix/Package.swift index 3450cc8..d6c1a3c 100644 --- a/examples/os-matrix/Package.swift +++ b/examples/os-matrix/Package.swift @@ -33,9 +33,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/plate-today-tools/Package.swift b/examples/plate-today-tools/Package.swift index ec3b0f1..2561840 100644 --- a/examples/plate-today-tools/Package.swift +++ b/examples/plate-today-tools/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" ), ] diff --git a/examples/plate-today/Package.swift b/examples/plate-today/Package.swift index 7e8751f..e7114bf 100644 --- a/examples/plate-today/Package.swift +++ b/examples/plate-today/Package.swift @@ -29,7 +29,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" ), ] diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index f098b38..c60daa6 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -37,9 +37,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/repo-qa/Package.swift b/examples/repo-qa/Package.swift index d540a92..890e023 100644 --- a/examples/repo-qa/Package.swift +++ b/examples/repo-qa/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" ), ] diff --git a/examples/security-demo/Package.swift b/examples/security-demo/Package.swift index 75211ef..824e524 100644 --- a/examples/security-demo/Package.swift +++ b/examples/security-demo/Package.swift @@ -38,7 +38,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "88f8145fbaff6ff9d9ec7332abb8837ae4010b2fbbe390527e18ac50fdc77ad3" + remoteChecksum: "013873523fcbb03aa1ed03e808fc90d440cd7dc1ca8ffcbc326a19cfe81d2337" ), ] diff --git a/examples/vistanova/Package.swift b/examples/vistanova/Package.swift index 6cc3fb5..9cb5324 100644 --- a/examples/vistanova/Package.swift +++ b/examples/vistanova/Package.swift @@ -35,9 +35,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/workspace-buddy-local/Package.swift b/examples/workspace-buddy-local/Package.swift index c45357d..112b16c 100644 --- a/examples/workspace-buddy-local/Package.swift +++ b/examples/workspace-buddy-local/Package.swift @@ -39,9 +39,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9", + coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "014bcbf373cfad17aaae2aaff9e9b5460e33512a7ef5db76864981f313dc5a63" + inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" ), ] diff --git a/examples/workspace-buddy/Package.swift b/examples/workspace-buddy/Package.swift index e15f501..5f05634 100644 --- a/examples/workspace-buddy/Package.swift +++ b/examples/workspace-buddy/Package.swift @@ -30,7 +30,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "ae20766fe20094b5703826143926f96a222cb98a0381291dac4c5cf9261c78c9" + checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" ), ] From dd673853b1147441e9197437c41889845182ea7d Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 06:20:07 -0700 Subject: [PATCH 22/73] examples: add Xcode project for mcp-chat XcodeGen spec (project.yml) and the generated, committed MCPChat.xcodeproj so the example opens in Xcode and Runs. Sandboxed with network.client, automatic signing, mcpchat: OAuth URL scheme. Package.swift vends LocalLMLabSDKInference as a library product for the project to depend on. README gains an "Open in Xcode and Run" section. Co-Authored-By: Claude Opus 5.5 --- .../MCPChat.xcodeproj/project.pbxproj | 387 ++++++++++++++++++ .../contents.xcworkspacedata | 7 + .../xcshareddata/xcschemes/MCPChat.xcscheme | 91 ++++ examples/mcp-chat/Package.swift | 6 + examples/mcp-chat/README.md | 15 +- examples/mcp-chat/project.yml | 83 ++++ .../mcp-chat/xcodeproj/MCPChat-Info.plist | 41 ++ .../mcp-chat/xcodeproj/MCPChat.entitlements | 10 + 8 files changed, 639 insertions(+), 1 deletion(-) create mode 100644 examples/mcp-chat/MCPChat.xcodeproj/project.pbxproj create mode 100644 examples/mcp-chat/MCPChat.xcodeproj/project.xcworkspace/contents.xcworkspacedata create mode 100644 examples/mcp-chat/MCPChat.xcodeproj/xcshareddata/xcschemes/MCPChat.xcscheme create mode 100644 examples/mcp-chat/project.yml create mode 100644 examples/mcp-chat/xcodeproj/MCPChat-Info.plist create mode 100644 examples/mcp-chat/xcodeproj/MCPChat.entitlements diff --git a/examples/mcp-chat/MCPChat.xcodeproj/project.pbxproj b/examples/mcp-chat/MCPChat.xcodeproj/project.pbxproj new file mode 100644 index 0000000..728308d --- /dev/null +++ b/examples/mcp-chat/MCPChat.xcodeproj/project.pbxproj @@ -0,0 +1,387 @@ +// !$*UTF8*$! +{ + archiveVersion = 1; + classes = { + }; + objectVersion = 77; + objects = { + +/* Begin PBXBuildFile section */ + 19481C628BA8411F5BB0A6CA /* WidgetSlot.swift in Sources */ = {isa = PBXBuildFile; fileRef = DA9542AF8CEFE77102E896CC /* WidgetSlot.swift */; }; + 27FD2FF257A2806A5D7E3A8C /* LocalLMLabSDKMCPAppsHost in Frameworks */ = {isa = PBXBuildFile; productRef = 66CFBD625CF9FF9BA9654C27 /* LocalLMLabSDKMCPAppsHost */; }; + 4D4992A147AF09C8B8A68B84 /* LocalLMLabSDKComponents in Frameworks */ = {isa = PBXBuildFile; productRef = 31E823BEFF3B7B5210B7E3AA /* LocalLMLabSDKComponents */; }; + 64B1506A1245B1B8AEC67A8A /* LocalLMLabSDKInference in Frameworks */ = {isa = PBXBuildFile; productRef = 0FDB2E05A68522B603B1F918 /* LocalLMLabSDKInference */; }; + 901B221AE12E7CA29BA2C849 /* ChatViews.swift in Sources */ = {isa = PBXBuildFile; fileRef = 04F9E13E6BE70043548EE4CB /* ChatViews.swift */; }; + A2318D2C044066A51DD846DD /* ChatModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = F3ACAA422499EB29B3FFF688 /* ChatModel.swift */; }; + B97B35364975636A5B4F674A /* MCPChatApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = 310454EC137FB5CCE7AEC492 /* MCPChatApp.swift */; }; + FAECE8892F0DE86CC905C28C /* LocalLMLabSDKCore in Frameworks */ = {isa = PBXBuildFile; productRef = 11E44880347842B5C305B499 /* LocalLMLabSDKCore */; }; +/* End PBXBuildFile section */ + +/* Begin PBXFileReference section */ + 04F9E13E6BE70043548EE4CB /* ChatViews.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ChatViews.swift; sourceTree = ""; }; + 310454EC137FB5CCE7AEC492 /* MCPChatApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = MCPChatApp.swift; sourceTree = ""; }; + 44A5B6BFCC90D814B643060E /* MCPChat.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = MCPChat.app; sourceTree = BUILT_PRODUCTS_DIR; }; + 48E121CCEAE6DF5221D6F479 /* Components */ = {isa = PBXFileReference; lastKnownFileType = folder; name = Components; path = ../../Components; sourceTree = SOURCE_ROOT; }; + 8E1A86286036B0DCB87DBF91 /* mcp-chat */ = {isa = PBXFileReference; lastKnownFileType = folder; name = "mcp-chat"; path = .; sourceTree = SOURCE_ROOT; }; + DA9542AF8CEFE77102E896CC /* WidgetSlot.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = WidgetSlot.swift; sourceTree = ""; }; + DF0A4BEB1B997BFAFD782603 /* MCPAppsHost */ = {isa = PBXFileReference; lastKnownFileType = folder; name = MCPAppsHost; path = ../../MCPAppsHost; sourceTree = SOURCE_ROOT; }; + F3ACAA422499EB29B3FFF688 /* ChatModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ChatModel.swift; sourceTree = ""; }; +/* End PBXFileReference section */ + +/* Begin PBXFrameworksBuildPhase section */ + D07C2745353B9C8C670FBEF0 /* Frameworks */ = { + isa = PBXFrameworksBuildPhase; + buildActionMask = 2147483647; + files = ( + FAECE8892F0DE86CC905C28C /* LocalLMLabSDKCore in Frameworks */, + 4D4992A147AF09C8B8A68B84 /* LocalLMLabSDKComponents in Frameworks */, + 27FD2FF257A2806A5D7E3A8C /* LocalLMLabSDKMCPAppsHost in Frameworks */, + 64B1506A1245B1B8AEC67A8A /* LocalLMLabSDKInference in Frameworks */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXFrameworksBuildPhase section */ + +/* Begin PBXGroup section */ + 2DB71789CDFE762BFFFD62CB = { + isa = PBXGroup; + children = ( + D167C6414C3115F3CDBC1821 /* Packages */, + 963CC54DEA6E910A0B116AA0 /* Sources */, + 6A0E4683C8F000DC6CA6AF2A /* Products */, + ); + sourceTree = ""; + }; + 40977F233C6F71D2F079D08E /* MCPChat */ = { + isa = PBXGroup; + children = ( + F3ACAA422499EB29B3FFF688 /* ChatModel.swift */, + 04F9E13E6BE70043548EE4CB /* ChatViews.swift */, + 310454EC137FB5CCE7AEC492 /* MCPChatApp.swift */, + DA9542AF8CEFE77102E896CC /* WidgetSlot.swift */, + ); + path = MCPChat; + sourceTree = ""; + }; + 6A0E4683C8F000DC6CA6AF2A /* Products */ = { + isa = PBXGroup; + children = ( + 44A5B6BFCC90D814B643060E /* MCPChat.app */, + ); + name = Products; + sourceTree = ""; + }; + 963CC54DEA6E910A0B116AA0 /* Sources */ = { + isa = PBXGroup; + children = ( + 40977F233C6F71D2F079D08E /* MCPChat */, + ); + path = Sources; + sourceTree = ""; + }; + D167C6414C3115F3CDBC1821 /* Packages */ = { + isa = PBXGroup; + children = ( + 48E121CCEAE6DF5221D6F479 /* Components */, + 8E1A86286036B0DCB87DBF91 /* mcp-chat */, + DF0A4BEB1B997BFAFD782603 /* MCPAppsHost */, + ); + name = Packages; + sourceTree = ""; + }; +/* End PBXGroup section */ + +/* Begin PBXNativeTarget section */ + 503EE6062818BDC97AE261E5 /* MCPChat */ = { + isa = PBXNativeTarget; + buildConfigurationList = C44D6E7D3DC00B84B0F8C503 /* Build configuration list for PBXNativeTarget "MCPChat" */; + buildPhases = ( + 3D29A9B3B3B0EF177B5C4993 /* Sources */, + D07C2745353B9C8C670FBEF0 /* Frameworks */, + ); + buildRules = ( + ); + dependencies = ( + ); + name = MCPChat; + packageProductDependencies = ( + 11E44880347842B5C305B499 /* LocalLMLabSDKCore */, + 31E823BEFF3B7B5210B7E3AA /* LocalLMLabSDKComponents */, + 66CFBD625CF9FF9BA9654C27 /* LocalLMLabSDKMCPAppsHost */, + 0FDB2E05A68522B603B1F918 /* LocalLMLabSDKInference */, + ); + productName = MCPChat; + productReference = 44A5B6BFCC90D814B643060E /* MCPChat.app */; + productType = "com.apple.product-type.application"; + }; +/* End PBXNativeTarget section */ + +/* Begin PBXProject section */ + 58721FE66928B4C294C129B1 /* Project object */ = { + isa = PBXProject; + attributes = { + BuildIndependentTargetsInParallel = YES; + LastUpgradeCheck = 1430; + TargetAttributes = { + 503EE6062818BDC97AE261E5 = { + ProvisioningStyle = Automatic; + }; + }; + }; + buildConfigurationList = EF15E58188E5201F7A3A9D8D /* Build configuration list for PBXProject "MCPChat" */; + developmentRegion = en; + hasScannedForEncodings = 0; + knownRegions = ( + Base, + en, + ); + mainGroup = 2DB71789CDFE762BFFFD62CB; + minimizedProjectReferenceProxies = 1; + packageReferences = ( + D44CBB7DD8B438699143142A /* XCLocalSwiftPackageReference "../../Components" */, + 7F5F055146FDD8B6C06B3A30 /* XCLocalSwiftPackageReference "../../MCPAppsHost" */, + 60620E9F75D0B05E3E19A77C /* XCLocalSwiftPackageReference "." */, + ); + preferredProjectObjectVersion = 77; + productRefGroup = 6A0E4683C8F000DC6CA6AF2A /* Products */; + projectDirPath = ""; + projectRoot = ""; + targets = ( + 503EE6062818BDC97AE261E5 /* MCPChat */, + ); + }; +/* End PBXProject section */ + +/* Begin PBXSourcesBuildPhase section */ + 3D29A9B3B3B0EF177B5C4993 /* Sources */ = { + isa = PBXSourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + A2318D2C044066A51DD846DD /* ChatModel.swift in Sources */, + 901B221AE12E7CA29BA2C849 /* ChatViews.swift in Sources */, + B97B35364975636A5B4F674A /* MCPChatApp.swift in Sources */, + 19481C628BA8411F5BB0A6CA /* WidgetSlot.swift in Sources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXSourcesBuildPhase section */ + +/* Begin XCBuildConfiguration section */ + 1FC39FEB0E3269E4D0FA6137 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_ENTITLEMENTS = xcodeproj/MCPChat.entitlements; + CODE_SIGN_STYLE = Automatic; + COMBINE_HIDPI_IMAGES = YES; + CURRENT_PROJECT_VERSION = 1; + ENABLE_APP_SANDBOX = YES; + ENABLE_HARDENED_RUNTIME = NO; + INFOPLIST_FILE = "xcodeproj/MCPChat-Info.plist"; + LD_RUNPATH_SEARCH_PATHS = ( + "@executable_path/../Frameworks", + ); + MARKETING_VERSION = 0.1.0; + PRODUCT_BUNDLE_IDENTIFIER = lab.locallm.mcpapps.chat; + SDKROOT = macosx; + }; + name = Debug; + }; + 5724B193D279ECA2BCF6E1D5 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_ENABLE_OBJC_WEAK = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; + ENABLE_NS_ASSERTIONS = NO; + ENABLE_STRICT_OBJC_MSGSEND = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_NO_COMMON_BLOCKS = YES; + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + MACOSX_DEPLOYMENT_TARGET = 27.0; + MTL_ENABLE_DEBUG_INFO = NO; + MTL_FAST_MATH = YES; + PRODUCT_NAME = "$(TARGET_NAME)"; + SDKROOT = macosx; + SWIFT_COMPILATION_MODE = wholemodule; + SWIFT_OPTIMIZATION_LEVEL = "-O"; + SWIFT_VERSION = 5.0; + }; + name = Release; + }; + 7AAD27502C53C6C63A7BEDC1 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_ENTITLEMENTS = xcodeproj/MCPChat.entitlements; + CODE_SIGN_STYLE = Automatic; + COMBINE_HIDPI_IMAGES = YES; + CURRENT_PROJECT_VERSION = 1; + ENABLE_APP_SANDBOX = YES; + ENABLE_HARDENED_RUNTIME = NO; + INFOPLIST_FILE = "xcodeproj/MCPChat-Info.plist"; + LD_RUNPATH_SEARCH_PATHS = ( + "@executable_path/../Frameworks", + ); + MARKETING_VERSION = 0.1.0; + PRODUCT_BUNDLE_IDENTIFIER = lab.locallm.mcpapps.chat; + SDKROOT = macosx; + }; + name = Release; + }; + 99FBFA5387DB031A9320050B /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_ENABLE_OBJC_WEAK = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = dwarf; + ENABLE_STRICT_OBJC_MSGSEND = YES; + ENABLE_TESTABILITY = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_DYNAMIC_NO_PIC = NO; + GCC_NO_COMMON_BLOCKS = YES; + GCC_OPTIMIZATION_LEVEL = 0; + GCC_PREPROCESSOR_DEFINITIONS = ( + "$(inherited)", + "DEBUG=1", + ); + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + MACOSX_DEPLOYMENT_TARGET = 27.0; + MTL_ENABLE_DEBUG_INFO = INCLUDE_SOURCE; + MTL_FAST_MATH = YES; + ONLY_ACTIVE_ARCH = YES; + PRODUCT_NAME = "$(TARGET_NAME)"; + SDKROOT = macosx; + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; + SWIFT_OPTIMIZATION_LEVEL = "-Onone"; + SWIFT_VERSION = 5.0; + }; + name = Debug; + }; +/* End XCBuildConfiguration section */ + +/* Begin XCConfigurationList section */ + C44D6E7D3DC00B84B0F8C503 /* Build configuration list for PBXNativeTarget "MCPChat" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 1FC39FEB0E3269E4D0FA6137 /* Debug */, + 7AAD27502C53C6C63A7BEDC1 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Debug; + }; + EF15E58188E5201F7A3A9D8D /* Build configuration list for PBXProject "MCPChat" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 99FBFA5387DB031A9320050B /* Debug */, + 5724B193D279ECA2BCF6E1D5 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Debug; + }; +/* End XCConfigurationList section */ + +/* Begin XCLocalSwiftPackageReference section */ + 60620E9F75D0B05E3E19A77C /* XCLocalSwiftPackageReference "." */ = { + isa = XCLocalSwiftPackageReference; + relativePath = .; + }; + 7F5F055146FDD8B6C06B3A30 /* XCLocalSwiftPackageReference "../../MCPAppsHost" */ = { + isa = XCLocalSwiftPackageReference; + relativePath = ../../MCPAppsHost; + }; + D44CBB7DD8B438699143142A /* XCLocalSwiftPackageReference "../../Components" */ = { + isa = XCLocalSwiftPackageReference; + relativePath = ../../Components; + }; +/* End XCLocalSwiftPackageReference section */ + +/* Begin XCSwiftPackageProductDependency section */ + 0FDB2E05A68522B603B1F918 /* LocalLMLabSDKInference */ = { + isa = XCSwiftPackageProductDependency; + productName = LocalLMLabSDKInference; + }; + 11E44880347842B5C305B499 /* LocalLMLabSDKCore */ = { + isa = XCSwiftPackageProductDependency; + productName = LocalLMLabSDKCore; + }; + 31E823BEFF3B7B5210B7E3AA /* LocalLMLabSDKComponents */ = { + isa = XCSwiftPackageProductDependency; + productName = LocalLMLabSDKComponents; + }; + 66CFBD625CF9FF9BA9654C27 /* LocalLMLabSDKMCPAppsHost */ = { + isa = XCSwiftPackageProductDependency; + productName = LocalLMLabSDKMCPAppsHost; + }; +/* End XCSwiftPackageProductDependency section */ + }; + rootObject = 58721FE66928B4C294C129B1 /* Project object */; +} diff --git a/examples/mcp-chat/MCPChat.xcodeproj/project.xcworkspace/contents.xcworkspacedata b/examples/mcp-chat/MCPChat.xcodeproj/project.xcworkspace/contents.xcworkspacedata new file mode 100644 index 0000000..919434a --- /dev/null +++ b/examples/mcp-chat/MCPChat.xcodeproj/project.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,7 @@ + + + + + diff --git a/examples/mcp-chat/MCPChat.xcodeproj/xcshareddata/xcschemes/MCPChat.xcscheme b/examples/mcp-chat/MCPChat.xcodeproj/xcshareddata/xcschemes/MCPChat.xcscheme new file mode 100644 index 0000000..be9882e --- /dev/null +++ b/examples/mcp-chat/MCPChat.xcodeproj/xcshareddata/xcschemes/MCPChat.xcscheme @@ -0,0 +1,91 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift index 84f8208..b22d959 100644 --- a/examples/mcp-chat/Package.swift +++ b/examples/mcp-chat/Package.swift @@ -56,6 +56,12 @@ guard let sdk = knownSDKReleases[requested] else { let package = Package( name: "MCPChat", platforms: [.macOS("27.0")], + products: [ + // Vend the Inference binary as a library product so the XcodeGen .xcodeproj variant + // (project.yml) can depend on it by name. `swift build` doesn't need this. Core comes from + // Components' own product, never re-declared here (see above). + .library(name: "LocalLMLabSDKInference", targets: ["LocalLMLabSDKInference"]) + ], dependencies: [ .package(path: "../../Components"), .package(path: "../../MCPAppsHost"), diff --git a/examples/mcp-chat/README.md b/examples/mcp-chat/README.md index 1ed88c3..1a33ae9 100644 --- a/examples/mcp-chat/README.md +++ b/examples/mcp-chat/README.md @@ -13,7 +13,20 @@ Release, `Components` (`../../Components`) and `MCPAppsHost` (`../../MCPAppsHost source. Needs macOS 27 and Xcode 27 (Swift 6.4). No Metal Toolchain needed — the prebuilt Inference xcframework bundles the compiled shaders. -## Build and run +## Open in Xcode and Run + +```bash +open -a Xcode MCPChat.xcodeproj +``` + +Pick the **MCPChat** scheme and Run. The app is App-Sandboxed with outbound network only, like the +packaged build. Signing is *Automatic* with no hard-coded team: add an Apple ID in **Xcode ▸ Settings +▸ Accounts** (free is enough), or switch Signing to **Sign to Run Locally**. + +Generated from [`project.yml`](project.yml) with [XcodeGen](https://github.com/yonaskolb/XcodeGen). +Edit `project.yml`, not the `.xcodeproj`, then run `xcodegen generate`. + +## Build and run from the command line ```bash packaging/build-and-sign.sh # sandboxed release build → dist/MCP Chat.app diff --git a/examples/mcp-chat/project.yml b/examples/mcp-chat/project.yml new file mode 100644 index 0000000..cd9ee99 --- /dev/null +++ b/examples/mcp-chat/project.yml @@ -0,0 +1,83 @@ +# XcodeGen spec for the "double-click, Run" variant of MCP Chat. +# +# brew install xcodegen && xcodegen generate # regenerates MCPChat.xcodeproj +# +# The generated MCPChat.xcodeproj IS committed — a fresh clone opens it and Runs. +# Edit project.yml, not the .xcodeproj, then regenerate. packaging/build-and-sign.sh remains the +# way to make a distributable .app. +# +# App-Sandboxed with outbound network only (MCP servers, OAuth, the one-time model download) — the +# same entitlements as packaging/MCPChat.entitlements. To use an existing model cache instead of +# downloading, turn App Sandbox off and set MCPCHAT_MODEL_CACHE in the scheme's Run environment. +# +# Info.plist is written by XcodeGen to xcodeproj/MCPChat-Info.plist (committed) from the +# `properties` below — separate from packaging/Info.plist, which build-and-sign.sh owns. The +# `mcpchat:` URL scheme (MCP OAuth redirect target) is carried here too. +# +# Signing: CODE_SIGN_STYLE = Automatic, no hard-coded team — resolves to your Apple Development +# identity in Xcode. None -> Run says "requires a development team" until you add an Apple ID +# (Xcode > Settings > Accounts, free is enough) or switch to "Sign to Run Locally". +name: MCPChat +options: + bundleIdPrefix: lab.locallm.mcpapps + deploymentTarget: + macOS: "27.0" + createIntermediateGroups: true + groupSortPosition: top +schemes: + MCPChat: + build: + targets: + MCPChat: all + run: + config: Debug +packages: + MCPChatPackage: + path: . + Components: + path: ../../Components + MCPAppsHost: + path: ../../MCPAppsHost +targets: + MCPChat: + type: application + platform: macOS + sources: + - path: Sources/MCPChat + info: + path: xcodeproj/MCPChat-Info.plist + properties: + CFBundleDisplayName: MCP Chat + CFBundleName: MCP Chat + LSApplicationCategoryType: public.app-category.developer-tools + LSMinimumSystemVersion: "27.0" + NSHighResolutionCapable: true + CFBundleURLTypes: + - CFBundleURLName: lab.locallm.mcpapps.chat.oauth + CFBundleURLSchemes: [mcpchat] + # XcodeGen owns (regenerates) this file — separate from packaging/MCPChat.entitlements, + # which build-and-sign.sh owns. + entitlements: + path: xcodeproj/MCPChat.entitlements + properties: + com.apple.security.app-sandbox: true + com.apple.security.network.client: true + settings: + base: + PRODUCT_BUNDLE_IDENTIFIER: lab.locallm.mcpapps.chat + MARKETING_VERSION: "0.1.0" + CURRENT_PROJECT_VERSION: "1" + CODE_SIGN_STYLE: Automatic + ENABLE_APP_SANDBOX: YES + ENABLE_HARDENED_RUNTIME: NO + LD_RUNPATH_SEARCH_PATHS: + - "@executable_path/../Frameworks" + dependencies: + - package: Components + product: LocalLMLabSDKCore + - package: Components + product: LocalLMLabSDKComponents + - package: MCPAppsHost + product: LocalLMLabSDKMCPAppsHost + - package: MCPChatPackage + product: LocalLMLabSDKInference diff --git a/examples/mcp-chat/xcodeproj/MCPChat-Info.plist b/examples/mcp-chat/xcodeproj/MCPChat-Info.plist new file mode 100644 index 0000000..c756f5c --- /dev/null +++ b/examples/mcp-chat/xcodeproj/MCPChat-Info.plist @@ -0,0 +1,41 @@ + + + + + CFBundleDevelopmentRegion + $(DEVELOPMENT_LANGUAGE) + CFBundleDisplayName + MCP Chat + CFBundleExecutable + $(EXECUTABLE_NAME) + CFBundleIdentifier + $(PRODUCT_BUNDLE_IDENTIFIER) + CFBundleInfoDictionaryVersion + 6.0 + CFBundleName + MCP Chat + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleURLTypes + + + CFBundleURLName + lab.locallm.mcpapps.chat.oauth + CFBundleURLSchemes + + mcpchat + + + + CFBundleVersion + 1 + LSApplicationCategoryType + public.app-category.developer-tools + LSMinimumSystemVersion + 27.0 + NSHighResolutionCapable + + + diff --git a/examples/mcp-chat/xcodeproj/MCPChat.entitlements b/examples/mcp-chat/xcodeproj/MCPChat.entitlements new file mode 100644 index 0000000..ee95ab7 --- /dev/null +++ b/examples/mcp-chat/xcodeproj/MCPChat.entitlements @@ -0,0 +1,10 @@ + + + + + com.apple.security.app-sandbox + + com.apple.security.network.client + + + From 832a14354fe7599e341bbc4825d45fc2fab63f93 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 19:17:17 -0700 Subject: [PATCH 23/73] 2.0 changelog and migration guide: local (MLX) model changes CHANGELOG 2.0: defaulted maxModelCallsPerTurn parameters (breaking, shape); real MLX usage, the 16-call tool-loop stop and up-front prompt refusals (behavior); new "Added - local (MLX) models" (MLXUsageMetadataKey, MLXPrefillMonitor, maxModelCallsPerTurn); new "Fixed" (contextBudget). migrating-to-2.0.md: the same behavior changes under "What behaves differently", including how to raise the call limit, and usage/prefill reporting under "What's new". Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 25 +++++++++++++++++++++++++ docs/migrating-to-2.0.md | 15 +++++++++++++++ 2 files changed, 40 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a7c2f11..2c08fbd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,6 +42,8 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) - **`MCPServerState`** gained fields (`trust`, `toolApproval`, `trustedToolsDigest`, `liveUpdates`, …): a state JSON encoded under 1.0 no longer decodes. Restore servers with `restore(from:)`. +- **Defaulted parameters added**: `SessionOptions.init` and `MLXModelProvider.init` gain + `maxModelCallsPerTurn:` (usual call sites compile unchanged). - **One declaration each** where 1.x had kept two (usual call sites compile unchanged): `makeSession` (now with `restoring:` and `mcpAppHints:`), `respond` / `streamResponse` (with `options:` and `fromAppInstance:`), `LocalLMLab.Configuration.init` (with `mcp: MCPSettings`), @@ -57,6 +59,15 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) models; `ui://` resources are left out of model-facing resource lists. - `languageModelSession` is documented as the escape hatch: run turns with `respond` / `streamResponse`. +- **Local (MLX) models report real usage**: input and output token counts are the model's own (input was always + 0). `languageModelSession.usage` sums every model call in the session; one turn's figures are on its response's + `usage`, with `mlx.*` metadata (`MLXUsageMetadataKey`). +- **An MLX turn stops after 16 model calls** — a model that keeps calling tools without answering — with an error + saying so. Raise or remove the limit with `MLXModelProvider(maxModelCallsPerTurn:)` or + `SessionOptions.maxModelCallsPerTurn` (`Int.max` = no limit). +- **An MLX prompt too long for the model's context window, or for the GPU memory left, is refused before it runs** + (`LocalLMLabError.context`). `respond` / `streamResponse` treat it as a context overflow, so + `retryOnContextOverflow` compacts and retries. - MCP servers that only speak `2024-11-05`, or use the old HTTP+SSE transport, fail `addServer` with `.protocolMismatch` (1.x connected to the former and failed on the latter with "HTTP 404"). @@ -133,6 +144,20 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) `issuer` and redirect `iss` are checked (RFC 8414, RFC 9207); a changed authorization server mid-connection drops the old credentials and signs in again. +### Added — local (MLX) models: usage, speed and memory per turn (`LocalLMLabSDKInference`) + +- **`MLXUsageMetadataKey`**: real prompt and generated token counts, prompt (pp) and generation (tg) speed, + seconds, model calls, stop reason, speculative-decoding draft stats, and MLX memory (resident, turn peak, GPU + limit) — turn totals on the response's `usage.metadata`. +- **`MLXPrefillMonitor` / `MLXPrefillActivity`**: the prompt a local model is reading right now — exact tokens, + which model call of the turn, since when — so an app can say why nothing has appeared yet. +- **`maxModelCallsPerTurn`** on `MLXModelProvider` and `SessionOptions`: the tool-loop limit, 16 by default. + +### Fixed + +- **`contextBudget`** reported the whole session's cumulative usage as the last turn's, so `fractionUsed` grew + every turn. It now reports the latest turn's last model call (the conversation as the model last read it). + ### Added — examples - [`mcp-chat`](examples/mcp-chat/): a chat with a local model (Qwen3 8B via MLX, or Apple's diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md index 9f03366..7d63572 100644 --- a/docs/migrating-to-2.0.md +++ b/docs/migrating-to-2.0.md @@ -69,6 +69,18 @@ affects code that silenced that error another way. resource lists. - **A widget's tool calls** (an MCP App view, §6) that need the user's answer ask once per view and tool; destructive calls and timeouts are asked again. Model calls are unchanged. +- **Local (MLX) models report real token counts.** `usage` input counts were always 0 in 1.0; now they're + the model's own, per model call. `languageModelSession.usage` sums the whole session — for one turn, read + the response's `usage`. +- **A local model that keeps calling tools stops after 16 model calls in a turn**, with an error saying so. + If your app's tasks legitimately need more, raise it: `MLXModelProvider(maxModelCallsPerTurn: 40)`, or per + session `SessionOptions(maxModelCallsPerTurn:)`; `Int.max` removes the limit. +- **A local-model prompt that can't fit** (over the context window, or the GPU memory left) is refused before + it runs, as `LocalLMLabError.context`. With `retryOnContextOverflow` set, it's compacted and retried like + any overflow. +- **`contextBudget` measures the latest turn**, not the whole session (1.0's `fractionUsed` grew every + turn). It's recorded by `respond` / `streamResponse`; turns run on `languageModelSession` directly aren't + counted. ### MCP protocol @@ -124,6 +136,9 @@ passed `restoring:` before `tools:`). Declare support with `MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps())`. Reference app: [`examples/mcp-chat`](../examples/mcp-chat/); walkthrough [`sdk-guide.md` §3f](sdk-guide.md#3f-mcp-apps-showing-a-servers-interactive-views). +- **Local (MLX) model usage, speed and memory per turn**: `MLXUsageMetadataKey` on the response's + `usage.metadata` (tokens, pp / tg, model calls, MLX memory), and `MLXPrefillMonitor` for "what is the model + reading right now" while nothing has appeared yet. - **Per-server trust and tool approval** (§3), shown on each row of `Components`' `MCPServerPickerView`, which now also says when none of a server's tools are on. - **MCP protocol `2026-07-28`**, the current revision. It is stateless: no session, no From 93d5fc3e029442b47943d1dc2db8f09f177fc30e Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 19:31:14 -0700 Subject: [PATCH 24/73] flush-sdk-cache: clear SwiftPM's artifact cache entries (they're files) SwiftPM stores each cached artifact as a zip file named after its URL, but the script only removed directories, so it reported "(nothing cached)" and a rebuild kept failing on the old checksum after a republish. Co-Authored-By: Claude Opus 5.5 --- scripts/flush-sdk-cache.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/flush-sdk-cache.sh b/scripts/flush-sdk-cache.sh index a826cdd..b635ec2 100755 --- a/scripts/flush-sdk-cache.sh +++ b/scripts/flush-sdk-cache.sh @@ -40,8 +40,9 @@ echo "== SwiftPM global artifact cache (ancientcomputing/locallm entries) ==" swiftpm_cache="$HOME/Library/Caches/org.swift.swiftpm/artifacts" if [ -d "$swiftpm_cache" ]; then found=0 + # Entries are zip files named after the URL (directories in older SwiftPM), so test -e, not -d. for dir in "$swiftpm_cache"/https___github_com_ancientcomputing_locallm_*; do - [ -d "$dir" ] || continue + [ -e "$dir" ] || continue found=1 echo " removing $(basename "$dir")" rm -rf "$dir" From bc0635438af316f5091b1fe1d4c6bff9c427ffa7 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 19:31:14 -0700 Subject: [PATCH 25/73] Cascade republished 2.0.0-dev SDK checksums (Core, Inference, Remote, Claude) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All four 2.0.0-dev xcframeworks were rebuilt and re-uploaded on 2026-10-04 02:22–02:26 UTC (Core and Inference now carry the decision API: question sets, answer sets, lab.evaluate, DecisionCalibration, OpenJevDecisionProvider(mlx:tunedWith:)). Values are the release assets' SHA-256 digests; Components and the 14 examples pinning them build against them. Co-Authored-By: Claude Opus 5.5 --- Components/Package.swift | 2 +- examples/README.md | 4 ++-- examples/aiql/Package.swift | 4 ++-- examples/code-buddy/Package.swift | 4 ++-- examples/mcp-chat/Package.swift | 2 +- examples/mlx-control-room/Package.swift | 4 ++-- examples/model-switch/Package.swift | 2 +- examples/os-matrix/Package.swift | 4 ++-- examples/plate-today-tools/Package.swift | 2 +- examples/plate-today/Package.swift | 2 +- examples/repo-qa-local/Package.swift | 4 ++-- examples/repo-qa/Package.swift | 2 +- examples/security-demo/Package.swift | 2 +- examples/vistanova/Package.swift | 4 ++-- examples/workspace-buddy-local/Package.swift | 4 ++-- examples/workspace-buddy/Package.swift | 2 +- 16 files changed, 24 insertions(+), 24 deletions(-) diff --git a/Components/Package.swift b/Components/Package.swift index 09ff1f5..833a913 100644 --- a/Components/Package.swift +++ b/Components/Package.swift @@ -33,7 +33,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" + checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" ), ] diff --git a/examples/README.md b/examples/README.md index 2ea6794..805fd4e 100644 --- a/examples/README.md +++ b/examples/README.md @@ -185,9 +185,9 @@ distinct `.xcframework.zip.sha256`; don't reuse Core's for another module): ```bash curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip.sha256 -# → 20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f +# → 532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKClaude-2.0.0-dev.xcframework.zip.sha256 -# → b19b8a35eb1081d4aae3d8a1f12aae6e8a93f11f72016ddca6b2c44aa06ff2f9 +# → fc01751adada70e60e6fc8ad6e74415712fe9371eb2b138d0ff56905a12addd0 ``` Then point `defaultSDKVersion` at it (works everywhere, Xcode included) or pass diff --git a/examples/aiql/Package.swift b/examples/aiql/Package.swift index a1f14a4..c7c896a 100644 --- a/examples/aiql/Package.swift +++ b/examples/aiql/Package.swift @@ -42,9 +42,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/code-buddy/Package.swift b/examples/code-buddy/Package.swift index 82117f6..534c83c 100644 --- a/examples/code-buddy/Package.swift +++ b/examples/code-buddy/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift index b22d959..be4d1bc 100644 --- a/examples/mcp-chat/Package.swift +++ b/examples/mcp-chat/Package.swift @@ -36,7 +36,7 @@ let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-dev": SDKRelease( inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/mlx-control-room/Package.swift b/examples/mlx-control-room/Package.swift index 1ef8875..fca63b9 100644 --- a/examples/mlx-control-room/Package.swift +++ b/examples/mlx-control-room/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/model-switch/Package.swift b/examples/model-switch/Package.swift index dea14eb..8caea66 100644 --- a/examples/model-switch/Package.swift +++ b/examples/model-switch/Package.swift @@ -37,7 +37,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "013873523fcbb03aa1ed03e808fc90d440cd7dc1ca8ffcbc326a19cfe81d2337" + remoteChecksum: "c9d193b5d8b0864b562ccbeee94ff5eb8915d5428f3c4dcc38c06585e737790b" ), ] diff --git a/examples/os-matrix/Package.swift b/examples/os-matrix/Package.swift index d6c1a3c..2d43baf 100644 --- a/examples/os-matrix/Package.swift +++ b/examples/os-matrix/Package.swift @@ -33,9 +33,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/plate-today-tools/Package.swift b/examples/plate-today-tools/Package.swift index 2561840..875ffb4 100644 --- a/examples/plate-today-tools/Package.swift +++ b/examples/plate-today-tools/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" + checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" ), ] diff --git a/examples/plate-today/Package.swift b/examples/plate-today/Package.swift index e7114bf..9e13565 100644 --- a/examples/plate-today/Package.swift +++ b/examples/plate-today/Package.swift @@ -29,7 +29,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" + checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" ), ] diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index c60daa6..8161c99 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -37,9 +37,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/repo-qa/Package.swift b/examples/repo-qa/Package.swift index 890e023..6c64e0e 100644 --- a/examples/repo-qa/Package.swift +++ b/examples/repo-qa/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" + checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" ), ] diff --git a/examples/security-demo/Package.swift b/examples/security-demo/Package.swift index 824e524..78e3ac9 100644 --- a/examples/security-demo/Package.swift +++ b/examples/security-demo/Package.swift @@ -38,7 +38,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "013873523fcbb03aa1ed03e808fc90d440cd7dc1ca8ffcbc326a19cfe81d2337" + remoteChecksum: "c9d193b5d8b0864b562ccbeee94ff5eb8915d5428f3c4dcc38c06585e737790b" ), ] diff --git a/examples/vistanova/Package.swift b/examples/vistanova/Package.swift index 9cb5324..2f7d11d 100644 --- a/examples/vistanova/Package.swift +++ b/examples/vistanova/Package.swift @@ -35,9 +35,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/workspace-buddy-local/Package.swift b/examples/workspace-buddy-local/Package.swift index 112b16c..96448e4 100644 --- a/examples/workspace-buddy-local/Package.swift +++ b/examples/workspace-buddy-local/Package.swift @@ -39,9 +39,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "e9ff0960f3cb2028023e550fc3f0e8621741bc92ae8c7ceac5c0ffd7fcc0fe88" + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" ), ] diff --git a/examples/workspace-buddy/Package.swift b/examples/workspace-buddy/Package.swift index 5f05634..661b7c8 100644 --- a/examples/workspace-buddy/Package.swift +++ b/examples/workspace-buddy/Package.swift @@ -30,7 +30,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "20ac14db027cc3cc2075210a79e393201d95a6b5f8a24b400031e947454da30f" + checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" ), ] From 06aa9e7294323ad6eba60ad1ddc1d1b9bfbe06ed Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 19:34:16 -0700 Subject: [PATCH 26/73] examples: add JevDK, the decision API (Jev) playground Write noul / choice / score questions, run them on a local MLX model (OpenJevDecisionProvider) and on hosted Jev (TypeSafe via OpenRouter, Featherless) side by side, score a batch against marked answers, fit calibration, append runs to a results CSV, import/export answer sets, and export the tested questions for an app (DecisionQuestionSet). GUIDE.md is a plain-English developer's guide to building with decision models. Package.swift declares Core, Remote and Inference binary targets for SDK 2.0.0-dev. Verified: release build, --check on Qwen3-4B-4bit and --check-remote on the Featherless demo. Co-Authored-By: Claude Opus 5.5 --- examples/README.md | 4 +- examples/jevdk/.gitignore | 3 + examples/jevdk/GUIDE.md | 354 +++++++++ examples/jevdk/Package.swift | 93 +++ examples/jevdk/README.md | 179 +++++ examples/jevdk/Sources/JevDK/AppModel.swift | 743 ++++++++++++++++++ .../jevdk/Sources/JevDK/Calibration.swift | 123 +++ .../Sources/JevDK/CalibrationSheet.swift | 139 ++++ examples/jevdk/Sources/JevDK/EditorView.swift | 156 ++++ examples/jevdk/Sources/JevDK/Fonts.swift | 12 + .../jevdk/Sources/JevDK/HostedBackends.swift | 99 +++ examples/jevdk/Sources/JevDK/JevDKApp.swift | 350 +++++++++ examples/jevdk/Sources/JevDK/Keychain.swift | 42 + .../jevdk/Sources/JevDK/ModelsSheet.swift | 205 +++++ examples/jevdk/Sources/JevDK/Presets.swift | 226 ++++++ examples/jevdk/Sources/JevDK/ResultsCSV.swift | 149 ++++ .../jevdk/Sources/JevDK/ResultsView.swift | 490 ++++++++++++ .../Sources/OpenJevKit/EditorModel.swift | 254 ++++++ .../Sources/OpenJevKit/ModelCatalog.swift | 28 + 19 files changed, 3648 insertions(+), 1 deletion(-) create mode 100644 examples/jevdk/.gitignore create mode 100644 examples/jevdk/GUIDE.md create mode 100644 examples/jevdk/Package.swift create mode 100644 examples/jevdk/README.md create mode 100644 examples/jevdk/Sources/JevDK/AppModel.swift create mode 100644 examples/jevdk/Sources/JevDK/Calibration.swift create mode 100644 examples/jevdk/Sources/JevDK/CalibrationSheet.swift create mode 100644 examples/jevdk/Sources/JevDK/EditorView.swift create mode 100644 examples/jevdk/Sources/JevDK/Fonts.swift create mode 100644 examples/jevdk/Sources/JevDK/HostedBackends.swift create mode 100644 examples/jevdk/Sources/JevDK/JevDKApp.swift create mode 100644 examples/jevdk/Sources/JevDK/Keychain.swift create mode 100644 examples/jevdk/Sources/JevDK/ModelsSheet.swift create mode 100644 examples/jevdk/Sources/JevDK/Presets.swift create mode 100644 examples/jevdk/Sources/JevDK/ResultsCSV.swift create mode 100644 examples/jevdk/Sources/JevDK/ResultsView.swift create mode 100644 examples/jevdk/Sources/OpenJevKit/EditorModel.swift create mode 100644 examples/jevdk/Sources/OpenJevKit/ModelCatalog.swift diff --git a/examples/README.md b/examples/README.md index 805fd4e..6d17496 100644 --- a/examples/README.md +++ b/examples/README.md @@ -22,6 +22,7 @@ Code samples for LocalLM Lab, split by which feature they use. | [security-demo/](security-demo/) | LocalLM Lab SDK (Core **+ Remote** + `Components`) | Tool authorization: a "Security" panel whose two controls are `Sequence.limited(toMaxImpact:)` (which tools the model sees) and `ConfirmingToolAuthorizer` + `Components.ToolConfirmationPresenter` (whether a call runs or asks first). A frontier model against a real Calendar and a real Todoist MCP server. | | [vistanova/](vistanova/) | LocalLM Lab SDK (Core **+ Inference**) | A tiny local search engine: a SwiftUI app that runs web search through a Tavily MCP server on one local model and summarizes on another — by default a downloaded MLX model **shipped pinned to an exact commit** (`MLXModelProvider(pinnedRevisions:)`), so a new upstream version is never picked up silently. Also shows a hand-written `Tool` over an MCP tool, and the defenses a small model needs: verify from `session.events` that it really called the tool, probe `capabilityProbe` before trusting structured output. | | [aiql/](aiql/) | LocalLM Lab SDK (Core **+ Inference**) | "Ask your data": a SwiftUI app that pulls an MCP-fronted dataset and writes the spreadsheet you asked for — `FileBackedTool` + the data verbs (`jsonToCsv` / `filterRows` / `sortRows` / …, guide §8b) do every row-level step so a local MLX model can't fabricate a value. Sandboxed; MCP OAuth via the `aiql://` URL scheme. | +| [jevdk/](jevdk/) | LocalLM Lab SDK (Core **+ Inference + Remote**), **2.0** | The decision API (Jev): a playground for the yes/no, choice and score questions an app asks a decision model. Runs them on a local MLX model (`OpenJevDecisionProvider`) and on hosted Jev (TypeSafe via OpenRouter, Featherless) side by side, scores a batch against your marked answers, fits calibration, appends runs to a results CSV for comparing models, and exports the tested questions for your app (`DecisionQuestionSet`). Comes with [a developer's guide](jevdk/GUIDE.md). | | [mcp-chat/](mcp-chat/) | LocalLM Lab SDK (Core **+ Inference** + `Components` + `MCPAppsHost`), **2.0** | MCP Apps in a chat: a local MLX model (or Apple's on-device model) whose tool calls show the server's interactive widget inline in the conversation — Todoist's task list, a system monitor. The widget's own calls go through the session, so they meet the same per-server trust / approval as the model's. Streaming replies, reasoning split out, conversations that survive relaunch. Sandboxed `.app`. | **Path A vs Path B**, for the four Core-based examples above: two ways to turn a connector or MCP @@ -71,6 +72,7 @@ build against the latest stable release. No environment variable is needed for e | `mlx-control-room` | SwiftUI + App Sandbox (network only) | ✅ via the committed `.xcodeproj` — **Automatic** signing (as above); it keeps no security-scoped bookmark, so **Sign to Run Locally** works too | **any** identity — `packaging/build-and-sign.sh` | | `components-updates-demo` | SwiftUI, no system permissions, simulated data | ✅ via `Package.swift` / `swift run` (no `.xcodeproj`) | — (no `packaging/`) | | `workspace-buddy`, `workspace-buddy-local`, `aiql` | SwiftUI + App Sandbox | ✅ via the committed `.xcodeproj` — **Automatic** signing (as above); a stable team identity is what lets the security-scoped bookmark survive a rebuild. Bare `swift run` is compile-only. | same as `plate-today` | +| `jevdk` | SwiftUI, no system permissions, no sandbox; hosted deciders optional (Featherless demo needs no key) | ✅ via `Package.swift` / `swift run -c release JevDK` (no `.xcodeproj`) | — (no `packaging/`) | | `mcp-chat` | SwiftUI + App Sandbox (network) | Compile-only via `Package.swift` / `swift build` (no `.xcodeproj`) | **any** identity, or **none** (`APP_IDENTITY=-`) — `packaging/build-and-sign.sh` | "Apple Development" = the free identity Xcode creates once you add any Apple ID under @@ -173,7 +175,7 @@ let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease(url: "…/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2…"), "2.0.0-dev": SDKRelease(url: "…/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "20ac14db…"), + checksum: "532bc89a…"), ] ``` diff --git a/examples/jevdk/.gitignore b/examples/jevdk/.gitignore new file mode 100644 index 0000000..ab886a9 --- /dev/null +++ b/examples/jevdk/.gitignore @@ -0,0 +1,3 @@ +.build/ +.env +key.env diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md new file mode 100644 index 0000000..8d86bd8 --- /dev/null +++ b/examples/jevdk/GUIDE.md @@ -0,0 +1,354 @@ +# Building with decision models: a developer's guide to JevDK + +This guide is for app developers who want a small model to make decisions inside their app: +route a message, pick a team, spot personal information, check an answer. It explains, in plain +English, how that differs from a chatbot, and how to use JevDK to get questions you can trust. + +You don't need to know anything about how models work inside. You do need some patience: a +decision model is closer to building a small test suite than to writing a prompt. + +--- + +## 1. What a decision model does + +A **chatbot** takes a prompt and writes a reply. You read the reply and judge it. + +A **decision model** ("decider") takes **one input**, such as a customer message, and answers +**a few fixed questions** about it. Each answer is one of the options you allowed, plus how sure +the model is. It writes no text. + +| Input | Question | Answer | +|---|---|---| +| "Nothing loads since this morning, my whole team is stuck." | Which team should handle this? (billing / technical / account) | technical, 100% | +| | Is the customer blocked from using the product right now? | Yes, 96% | +| | Does the customer explicitly ask for money back? | No, 97% | + +Your app then acts on the answers in ordinary code: send it to the technical team, mark it +urgent, don't open a refund. Nothing is generated, so it's fast (about 150 ms for four questions +on a small local model) and the answers are always something your code can switch on. + +The SDK call looks like this: + +```swift +let d = try await lab.decide(route: "support", state: .text(message), questions: [ + .choice("team", "Which team should handle this customer message?", criteria: [ + "billing": "payments, invoices and refunds", + "technical": "bugs, outages and product errors", + "account": "profile, login and subscription changes", + ]), + .noul("blocked", "Is the customer blocked from using the product right now?"), + .noul("refund", "Does the customer explicitly ask for money back?"), +]) +if let team = d.choice("team"), team.confidence > 0.8 { route(to: team.choice) } +``` + +### Why this is harder than a chatbot + +With a chatbot you judge each reply as you read it. With a decider, nobody reads the answers: +your code acts on them, thousands of times, unseen. So you can't judge a question by trying it +once. You have to know **how often it's right**, on inputs like the ones your app will see. That +means collecting examples, writing down the right answers, and measuring. JevDK is the tool for +that. + +--- + +## 2. When a decider is a good fit + +Good fits: questions a careful person could answer from the input alone, quickly and +consistently. + +- Routing: which team, which tool, which kind of request. +- Spotting things: personal information, a refund request, an angry customer, spam. +- Checking: does this answer address the question? Is this review about shipping? + +Poor fits: + +- **Anything that needs current facts.** The model has no internet and its knowledge stops at + its training date. "Does this query need live data?" is fine: the wording shows it. + "Is this price still correct?" isn't. +- **Writing or explaining.** That's the generator's job (a chat model), after the decider has + routed the request. +- **Judgements even people would disagree on.** If two colleagues wouldn't give the same answer, + the model won't be consistent either. + +--- + +## 3. The five things that decide your results + +1. **The model.** Different models are good at different questions. You find out by measuring, + not by reading a leaderboard. +2. **How each question is worded.** The single biggest lever. One word can flip answers. +3. **The system instructions** (local models only): the text that frames every question. Also + changes answers; JevDK lets you edit it, then your app uses the same text. +4. **Your answer set:** example inputs with the right answers written down. Without it you are + guessing. +5. **Calibration:** making the model's "how sure" numbers mean what they say. Optional, and + only once the answers themselves are good. + +The workflow below works through them in that order. + +--- + +## 4. The workflow in JevDK + +### Step 1. Write your questions + +On the left, **Add question** and pick a type: + +- **Noul (yes / no):** "Does the customer explicitly ask for money back?" Best for spotting + things. +- **Choice:** one of several options, each with a short description ("billing: payments, + invoices and refunds"). Best for routing. +- **Score:** a point on a scale (0 none … 3 deep). Small models are weakest here; use it only + when you truly need a scale, and consider a yes/no question at the threshold you care about + instead. + +Give each question a short **name** (what your code reads, e.g. `refund`) and write the +**question** the model reads. + +Wording rules that held up in testing: + +- **Ask about what the text says,** not what you'd infer. "Does the customer *explicitly* ask + for money back?" beats "Does the customer want a refund?" +- **Put context in the question.** "This post was made on a gardening forum. Which problem, if + any, does it have?" Hosted deciders never see the system instructions, so context there is + lost on them. +- **Describe choice options**, briefly. "code" is vague; "writing, fixing or explaining code or + software" isn't. +- **Plain and direct.** "Does this user query need live data?" worked better than longer, more + careful wordings on a small model. +- **One thing per question.** "Is the customer angry or threatening to leave?" mixes two; split + it if they matter separately. + +### Step 2. Try single inputs + +In **Single input**, type a realistic input and press **Ask** (⌘↩). Every question is answered +independently (no question sees another's answer). For each, you get a bar per option, the +chosen answer and how sure it was. + +What to look at: + +- **Is the answer right?** Obviously. But don't stop at a few successes; one input proves + little. +- **"Off format" warnings.** The model wanted to answer something other than your options. The + question or the options need rewording. +- **Show prompt** shows exactly what the local model read. If a question behaves oddly, read it + as the model does. + +Use this step to shape questions. Use the next steps to trust them. + +### Step 3. Build your answer set + +Collect **50 to 150 real inputs** like the ones your app will see. Real is better than invented: +real messages are messier, and the mess is where models go wrong. Include: + +- the common cases (most of your traffic), +- the awkward ones you already know about ("fix my invoice" is not a refund request), +- a few that should clearly be "none" or "no". + +Paste them into **Batch**, one per line, and **Ask all**. If you already have them in a +spreadsheet, save it as CSV with an `input` column and one column per question holding the +correct answer (blank where you haven't decided), and use **Answer set → Import** instead: the +inputs and your marks load together. Yes/no answers can be Yes/No, true/false or 1/0; a choice +is its key; a score is its level number or text. Then, under each cell, use **mark…** +to record the correct answer. This is the slow part, and the most valuable: from now on every +run is scored against your answers. + +Tips: + +- Mark answers as **you** want your app to behave, not as you guess the model will. +- If you hesitate over a mark, the question is probably ambiguous. Note it; you'll reword it in + step 5. +- Save the set (⌘S), and **Answer set → Export** the inputs and marks as CSV. That CSV is your + test suite: keep it with your app and grow it when you find new failure cases. JevDK also + remembers your workspace between launches. + +### Step 4. Run the batch and read the grid + +Each row is an input, each column a question, each cell an answer. With answers marked, every +backend's line shows **✓** or **✗**, its answer and how sure it was. Tinted cells mean backends +disagree. Hide the questions panel (sidebar button, top left) to see all the columns. + +Read it in this order: + +1. **Which columns have the ✗s?** A question that's wrong often needs rewording, not a different + model. +2. **Are the misses confident?** "✗ No 100%" is worse than "✗ No 55%": the second can be caught + with a threshold, the first can't. +3. **Do several models miss the same row?** Then check your mark and the wording first. In our + test, every model said a customer whose mobile app crashed (desktop fine) was "blocked". That + says more about the question than the models. + +### Step 5. Fix the questions, then run again + +Change one thing, **Ask all** again, compare. Typical fixes: + +- Make the question literal: "blocked from using the product" → "Does the message say the + customer cannot use the product at all right now?" +- Add or sharpen an option description. +- Split a question that mixes two things. +- Use **Duplicate** on a question card to run two wordings side by side and keep the better one. + +Stop when the remaining misses are ones you can live with, or can catch (step 8). + +### Step 6. Compare models + +The model is the answer to "which of these works best **on my questions**", and only your answer +set can tell you. JevDK doesn't keep a leaderboard on purpose. + +1. **Models** (toolbar): download a few candidates. Small dense instruct models suit deciding; + mixture-of-experts models are flagged because their probabilities were unstable in testing. +2. **Backends** (toolbar): optionally add a hosted decider (Featherless, or TypeSafe on + OpenRouter) as a reference point. +3. Run the batch with model A, then **Results CSV** → append. Switch to model B, run, append. +4. Open the CSV in Numbers or Excel. A pivot table of `correct` by `model` and `question` is + per-question accuracy; `question_ms` and `decision_ms` are speed. + +Weigh up: + +| | Why it matters | +|---|---| +| Accuracy per question | The question your app relies on most matters most. | +| Confidence when wrong | Low confidence on misses means a threshold can catch them. | +| Speed | Added to every request your app routes. | +| Size | A local decider sits next to your chat model in memory. | +| Local vs hosted | Local is private, free and offline; hosted can be more accurate and better calibrated. | + +### Step 7. Calibrate the model you picked (local only) + +A small model often says "100%" even when wrong; another might say "60%" when it's nearly +always right. **Calibration** adjusts the numbers so "80% sure" means right about 80% of the +time. It never changes which answer wins. + +You need it when your app acts on the confidence (thresholds, second opinions). You don't if you +only use the top answer. + +1. Have at least **30 marked answers per question type** you use (yes/no, choice, score). +2. **Calibrate…** → **Fit calibration.** JevDK finds one adjustment ("temperature") per question + type. Above 1 means the model was over-confident and gets softened; below 1, under-confident + and gets sharpened. +3. Read the before → after table. Lower is better on both measures (see section 6). +4. Turn on **Show local results calibrated** to see what your app will see. +5. **Export for App…** (step 8) carries it into your app with the questions. If you change the + model or the system instructions later, JevDK flags the calibration until you redo it. + +What calibration can't do: fix wrong answers. In our test, a confidently wrong answer stayed +wrong; it just became less confidently wrong. Get the questions right first (steps 4 and 5). + +### Step 8. Ship it + +Use the same questions, model, system instructions and calibration you tested. **File → +Export for App…** writes them into one file (`support.decisions.json`); add it to your app and +load it: + +```swift +let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "support.decisions", withExtension: "json")!) +let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // the tested wrapper and calibration +try lab.models.register(decision: openjev) +lab.models.route(decision: "support", to: ModelID("openjev:mlx-community/Qwen3-4B-4bit")!) + +let d = try await lab.decide(route: "support", state: .text(message), questions: set.questions) +``` + +Your app now asks exactly the questions you tested; to change one, change it in JevDK, test, +and export again. `openjev.tuningMismatch(for:)` tells you if the model on the device isn't the +one you tested (e.g. a newer revision). + +**Keep testing in your app's CI.** With the exported answer set, `lab.evaluate` runs every +example and scores it, so a model, SDK or question change that breaks answers fails a test: + +```swift +let answers = try DecisionAnswerSet(csv: String(contentsOf: answersURL, encoding: .utf8)) +let result = try await lab.evaluate(route: "support", questions: set.questions, answerSet: answers) +#expect(result.score("refund")!.accuracy >= 0.95) +``` + +Patterns that work: + +- **A threshold:** act on the answer when it's sure; otherwise do something safer (ask a + person, ask a bigger model, take the cautious path). In our test, "below 90%, get a second + opinion" would have flagged 7 of 64 answers and caught all three mistakes of one model. +- **Local first, hosted second:** answer easy questions locally; send only the hard or unsure + ones to a hosted decider. Your app decides the split (one `decide` call per backend). +- **Privacy gate first:** ask "does this contain personal information?" locally, and send + nothing to a hosted service unless it says no. +- **Keep the decider loaded:** `lab.models.pair(decision:generator:)` keeps the decider and + your chat model in memory together, so routing every message doesn't reload a model. +- **Keep your answer set:** re-run it (in JevDK or with `lab.evaluate`) whenever you change a + question, the model or the instructions, and when you upgrade the SDK. +- **Keep inputs short:** a decider needs the message, not the whole conversation. Send the last + few turns; hosted services limit input size (`DecisionLimits.maxStateCharacters`). + +--- + +## 5. A worked example: customer support + +Sixteen made-up customer messages, four questions (team, blocked, refund, angry), correct +answers marked, run on three deciders. Real results, 2026-10-03: + +| | team | blocked | refund | angry | Total | +|---|---|---|---|---|---| +| Featherless Qwen3.8-27B (hosted) | 15/16 | 15/16 | 16/16 | 15/16 | 61/64 | +| Featherless Qwen3.5-4B (hosted) | 16/16 | 15/16 | 15/16 | 15/16 | 61/64 | +| Qwen3-4B on this Mac | 15/16 | 12/16 | 16/16 | 15/16 | 58/64 | + +What a developer learns from this: + +- **Same total, different weaknesses.** The two hosted models both scored 61/64 but failed on + different messages. Totals hide what matters; look per question. +- **The local model's weak spot is one question.** Four of its six misses are `blocked`. That + points at rewording that question (step 5) or sending just that question to a hosted + decider (step 8), not at abandoning the local model. +- **Disagree with every model? Check yourself.** All three said the mobile-crash customer + (desktop works) was blocked. Arguably they're right and the question is ambiguous. +- **Confidence behaved differently per model.** The 27B was about 98% sure when right and 70% + when wrong: thresholds work well. The local 4B said 100% on everything, right or wrong; after + calibration it was 93% on right answers and 83% on wrong ones: more honest, but still not a + clean separation. That's a sign the `blocked` question needs work more than calibration does. + +Sixteen messages are enough to show the method, not to choose a model. Use 50 or more. + +--- + +## 6. Reading the numbers + +- **Confidence:** how sure the model says it is about its top answer. Not the same as how often + it's right, unless it's calibrated. +- **Accuracy:** share of answers matching your marks. The headline number, per question. +- **Calibration error:** how far "how sure it says" is from "how often it's right", averaged. + 0 is perfect; 0.05 is good; 0.15 means its percentages are misleading. +- **Log-loss:** punishes being confidently wrong. Lower is better. A big drop after calibration + means the model was over-confident. +- **On format (label mass):** how much of the model's probability went to your options at all. + Below 90% means it wanted to say something else: reword. + +--- + +## 7. Common mistakes + +- **Judging a question by one input.** Always run the batch. +- **Inventing tidy test inputs.** Real inputs fail in ways invented ones don't. +- **Asking for facts the model can't know.** Keep questions answerable from the input. +- **Calibrating to fix wrong answers.** It doesn't; fix the wording or the model. +- **Changing the model or instructions without re-running.** Answers and calibration are tied + to both. +- **Putting context in the system instructions and using a hosted decider.** It never sees them. +- **Trusting the total.** Look per question, and at how sure the misses were. +- **Using a mixture-of-experts model as the decider.** Use it as the chat model instead. + +--- + +## 8. Glossary + +- **Decider / decision model:** a model that answers fixed questions with one of your options + and a probability, instead of writing text. +- **Noul, Choice, Score:** the three question types: yes/no, one of several options, a point on + a scale. (Names from TypeSafe's Jev.) +- **Answer set:** example inputs with the correct answers marked. Your test suite. +- **Backend:** where the decider runs: locally (OpenJev on this Mac) or hosted (Featherless, + TypeSafe). +- **System instructions / wrapper:** the text that frames every question for a local model. +- **Calibration / temperature:** a per-question-type adjustment that makes confidence match how + often the model is right. +- **Batch:** the same questions asked about many inputs at once. +- **Results CSV:** the file JevDK appends each batch run to, for comparing models over time. diff --git a/examples/jevdk/Package.swift b/examples/jevdk/Package.swift new file mode 100644 index 0000000..ff27e76 --- /dev/null +++ b/examples/jevdk/Package.swift @@ -0,0 +1,93 @@ +// swift-tools-version: 6.4 +import Foundation +import PackageDescription + +// JevDK — a playground for decision questions. Write noul, choice or score questions, run them +// on a local MLX model and on hosted Jev, and see the answer probabilities, the exact prompt or +// request, and a batch grid across many inputs. See README.md and GUIDE.md. +// +// Built entirely on the SDK's decision API: local runs use LocalLMLabSDKInference's +// `OpenJevDecisionProvider`, hosted runs LocalLMLabSDKRemote's `JevDecisionProvider` through +// `lab.decide`, and models are listed, checked and downloaded by `MLXModelProvider`. OpenJevKit +// holds the playground's own editor and display types. +// +// Links three SDK binaries, all from one GitHub Release per version: LocalLMLabSDKCore, +// LocalLMLabSDKRemote and LocalLMLabSDKInference (the MLX runtime). It doesn't use Components, +// so it declares Core itself. Same SDKRelease/knownSDKReleases/failManifest version gate as every +// other example here. +// +// Requires SDK 2.0.0-dev or later (the decision API is new in 2.0), macOS 27 + Xcode 27. No Metal +// Toolchain needed — the prebuilt Inference xcframework bundles the compiled default.metallib. + +struct SDKRelease { + let coreURL: String + let coreChecksum: String + let remoteURL: String + let remoteChecksum: String + let inferenceURL: String + let inferenceChecksum: String +} + +// The SDK release this example builds against with no setup. Build against another published +// version: set LOCALLM_SDK_VERSION in your shell (works for `swift build` / CI, NOT inside Xcode), +// or edit `defaultSDKVersion` here. For a release not listed, add its entry (URL + the `.sha256` +// next to each zip on the GitHub release) or just replace the strings in place. Only 2.0 +// releases: this app does not build against 1.0. +let defaultSDKVersion = "2.0.0-dev" + +let knownSDKReleases: [String: SDKRelease] = [ + "2.0.0-dev": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", + coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", + remoteChecksum: "c9d193b5d8b0864b562ccbeee94ff5eb8915d5428f3c4dcc38c06585e737790b", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", + inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + ), +] + +func failManifest(_ message: String) -> Never { + FileHandle.standardError.write(Data((message + "\n").utf8)) + exit(1) +} + +let requested = ProcessInfo.processInfo.environment["LOCALLM_SDK_VERSION"] ?? defaultSDKVersion +guard let sdk = knownSDKReleases[requested] else { + failManifest(""" + error: Unknown LOCALLM_SDK_VERSION "\(requested)". + Known versions: \(knownSDKReleases.keys.sorted().joined(separator: ", ")) + """) +} + +let package = Package( + name: "JevDK", + platforms: [.macOS("27.0")], + products: [ + .library(name: "OpenJevKit", targets: ["OpenJevKit"]), + ], + targets: [ + .binaryTarget(name: "LocalLMLabSDKCore", url: sdk.coreURL, checksum: sdk.coreChecksum), + .binaryTarget(name: "LocalLMLabSDKRemote", url: sdk.remoteURL, checksum: sdk.remoteChecksum), + .binaryTarget(name: "LocalLMLabSDKInference", url: sdk.inferenceURL, checksum: sdk.inferenceChecksum), + .target( + name: "OpenJevKit", + dependencies: ["LocalLMLabSDKCore"], + swiftSettings: [.swiftLanguageMode(.v5)] + ), + .executableTarget( + name: "JevDK", + dependencies: [ + "OpenJevKit", + "LocalLMLabSDKCore", + "LocalLMLabSDKRemote", + "LocalLMLabSDKInference", + ], + swiftSettings: [.swiftLanguageMode(.v5)], + linkerSettings: [ + // Bare SwiftPM executables get no LC_RPATH; SwiftPM extracts the xcframework + // slices next to the built binary. Same fix as code-buddy / model-switch. + .unsafeFlags(["-Xlinker", "-rpath", "-Xlinker", "@executable_path"]) + ] + ), + ] +) diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md new file mode 100644 index 0000000..e5c4f38 --- /dev/null +++ b/examples/jevdk/README.md @@ -0,0 +1,179 @@ +# JevDK + +A playground for **decision questions**: try out the questions your app will ask a decision +model (a "decider"), and see how it answers before you put them in code. + +```bash +swift build -c release +.build/release/JevDK +``` + +Needs an Apple-silicon Mac, macOS 27 and Xcode 27 (Swift 6.4). `Package.swift` builds against SDK +`2.0.0-dev` with no further setup (the decision API is new in 2.0): `LocalLMLabSDKCore`, +`LocalLMLabSDKRemote` and `LocalLMLabSDKInference` (the MLX runtime) come from that GitHub Release. +No Metal Toolchain needed; the prebuilt Inference xcframework bundles the compiled shaders. To +build against another release, set `LOCALLM_SDK_VERSION` (see [the examples README](../README.md)). + +**New to decision models?** Read [the developer's guide](GUIDE.md) first: what a decider is, +when to use one, and the workflow from writing questions to shipping them, with a worked example. + +## The idea + +A chat model takes one prompt and writes one reply. + +A decider takes **one input** (a customer message, a user query, a review) and answers **a few +fixed questions** about it. Each answer is one of the options you allowed, plus how sure the model +is. It writes no text. + +| Input | Question | Answer | +|---|---|---| +| "Nothing loads since this morning, my whole team is stuck." | Which team should handle this? (billing / technical / account) | **technical** 100% | +| | Is the customer blocked from using the product right now? | **Yes** 96% | +| | Does the customer explicitly ask for money back? | **No** 97% | + +Your app then acts on the answers in plain code: send it to the technical team, mark it urgent, +don't open a refund. + +**Batch** asks the same questions about many inputs at once. Think of a spreadsheet: one row per +input, one column per question, each cell an answer. That's where you see the pattern: the +question that's right on nine messages and wrong on the tenth, or the wording that works better +than another. + +## Two-minute walkthrough + +1. **Examples → Customer support.** The left side fills with four questions. The right side + has an input box with sample messages above it. +2. **Click a sample** ("Double charge", "App outage", …). It runs straight away; each question + shows its answer, how sure the model is, and a bar for every option. +3. **Change a question** on the left, e.g. reword "Is the customer blocked…", and click the + same sample again to see if the answer moves. +4. **Switch to Batch** and press **Ask all**. Every sample runs; the grid shows all the answers + at once. Hover a cell to see every option's probability. +5. **Backends** (toolbar) turns on hosted deciders next to the local model, so each cell shows + one answer per decider, tinted where they disagree. + +**JevDK remembers your workspace**, like Prompt Playground keeps your last prompt: the +questions and system instructions, the input and batch list, your marked answers, calibration +and the selected model come back on the next launch. Switch the model or a backend, press +**Ask** or **Ask all**, and you're re-testing the exact same scenario. Results aren't kept; run +again. **Examples** asks before replacing work you've changed, and can save it to a file first. + +Your own questions: **Add question** on the left. Pick **Noul** (yes / no), **Choice** (one of +several options) or **Score** (a scale). Give it a short name (what your app reads), the +question text (what the model reads), and for Choice / Score the options. **Duplicate** a question +to try a second wording side by side. **⌘S / ⌘O** saves and opens a question set with its inputs. + +## Comparing models: the results CSV + +JevDK doesn't tell you which model to use; your questions and your answers do. After a batch, +**Results CSV** appends the run to a CSV file: one row per input × question × backend. Run the +same batch with another model (or quantization, wrapper or question wording) and append again; +the runs line up in one spreadsheet. The first click asks for a file; later clicks append to it +(**Append to another CSV…** to switch). A file with a different header is never touched. + +| Columns | | +|---|---| +| Run | `run_id` (date and time the batch started, plus a short suffix), `saved_at`, `app`, `sdk_version`, `machine`, `question_set` | +| Model | `backend` (local / featherlessDemo / featherless / openRouter), `model`, `model_revision`, `model_size_gb`, `moe`, `wrapper_label`, `wrapper_system`, `calibrated` | +| Item | `input`, `question`, `question_kind`, `question_text` | +| Result | `expected` (your marked answer), `answer`, `correct` (yes / no), `confidence`, `probabilities`, `label_mass` | +| Cost | `question_ms`, `decision_ms`, `input_tokens`, `cost_usd` (hosted, per decision) | + +Mark correct answers before saving so `expected` and `correct` are filled; then a pivot table +of `correct` by `model` and `question` is your per-question accuracy, and `question_ms` / +`decision_ms` your speed. Headless: add `--csv ` to `--check` or `--check-remote`. + +## Calibrating for your app + +A small local model often says 100% even when it's wrong. Calibration softens its confidence so +"80% sure" means right about 80% of the time; it never changes which answer wins. It depends on +the model, its version, the system instructions and your kind of questions, so you fit it here, +for your app: + +1. **Batch** with the local backend on: paste a few dozen inputs like the ones your app will see, + **Ask all**. +2. Under each cell, **mark…** the correct answer. Every backend's line then shows ✓ or ✗, so + you also get each backend's accuracy on your own questions. +3. **Calibrate…** → **Fit calibration**. It shows the calibration error and log-loss before → + after for each kind of question (aim for 30+ marks per kind). +4. Turn on **Show local results calibrated** to see what your app will see. It's saved with + the question set (⌘S). If you change the model or the system instructions, JevDK flags the + calibration until you refit. + +## Taking it to your app + +- **File → Export for App…** writes a `DecisionQuestionSet` JSON file: the questions plus the + model, revision, wrapper and calibration you tested. Bundle it and load it with + `DecisionQuestionSet(contentsOf:)`; `OpenJevDecisionProvider(mlx:tunedWith:)` applies the + wrapper and calibration. +- **Answer set → Export** (Batch toolbar, or the File menu) saves your inputs and marks as CSV: + an `input` column and one column per question. **Import** loads one, e.g. from a spreadsheet. + In your app's tests, `lab.evaluate(route:questions:answerSet:)` scores a backend against it. + +See [GUIDE.md](GUIDE.md) step 8. + +## Reading the results + +- **The percentage** is how sure the model is, not how likely it is to be right. Small local + models often say 100% when they're wrong. TypeSafe's hosted Jev is the only decider so far + whose numbers behave like real probabilities. +- **"Off format"** means the model wanted to answer something other than the options you + allowed. The question probably needs rewording. +- **Show prompt / Show request** shows exactly what the decider received. +- **Timing** per input is roughly what a router would add to each user turn. + +## Writing questions that work + +From the SDK's own OpenJev evaluation (a labelled test set scored on several local models): + +- Prefer **Noul** and **Choice**. Small models are much worse on **Score** scales. +- Give choice options a **short description**, not just a name. +- Plain, direct questions work well ("Does this user query need live data?"). +- Ask about **what the text says**: "Does the customer *explicitly* ask for money back?" beats + "Does the customer want a refund?" +- Only ask what a model can judge from the input and **general knowledge**: nothing that + depends on current events. +- **Put context in the question.** Hosted deciders don't see the system instructions, so "This + post was made on a gardening forum. Which problem does it have?", not a forum description in + the instructions. +- Check a wording on the model you'll ship. Different models prefer different wordings. + +## Choosing a decider + +- **Local** (toolbar picker; **Models** to change): MLX models the SDK has downloaded and + verified. **Models** has three tabs: **Installed** (use or remove), **Recommended** (small + dense instruct models, which ones were tested as deciders) and **Search Hugging Face** (MLX + models, most downloaded first). Select one, **Check** (the SDK confirms the repo is reachable, + MLX format, a supported architecture, and fits this Mac's memory and disk), then **Download**: + the SDK downloads and verifies it. Copies another tool put in the cache are offered with + **Verify** instead. `Qwen3-4B-4bit` is the current pick: about 150 ms for four questions, + 2.6 GB. Mixture-of-experts models are flagged; use them for chat, not for deciding. +- **Hosted** (Backends): the **Featherless demo** is free and needs no key (rate-limited); + **Featherless** and **OpenRouter · TypeSafe** take a key, stored in the Keychain. When any is + on, the status bar says where inputs are sent. + +## For SDK developers + +JevDK is built only on the SDK's public decision API, so it's also a reference client for it: + +- **Local:** LocalLMLabSDKInference's `OpenJevDecisionProvider`, wrapping `MLXModelProvider` + (which also lists, checks and downloads models). JevDK builds it with the editor's wrapper + (**System instructions** and what the input is called), so the wrapper stays editable here; + apps get `OpenJevWrapper.default`, chosen with the SDK's evaluation. Local runs call + `decideWithDiagnostics` for **Show prompt** and the off-format warning, after validating + against the provider's limits as `lab.decide` would. +- **Hosted:** LocalLMLabSDKRemote's `JevDecisionProvider` through `lab.decide`. +- `OpenJevKit` holds only the playground's own types: the editor model, saved files (earlier + yes/no / scale names still open), and result rows. + +Headless checks: + +```bash +.build/release/JevDK --check "Customer support" mlx-community/Qwen3-4B-4bit +.build/release/JevDK --check-remote "Customer support" openRouter +``` + +`--check` needs a model the SDK has downloaded (or verified). `--check-remote` takes +`featherlessDemo`, `featherless` or `openRouter`, with keys from the environment or a +git-ignored `./.env` (`OPENROUTER_API_KEY`, `FEATHERLESS_API_KEY`), never printed. The window +hasn't been driven by a test. diff --git a/examples/jevdk/Sources/JevDK/AppModel.swift b/examples/jevdk/Sources/JevDK/AppModel.swift new file mode 100644 index 0000000..6ebdab4 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/AppModel.swift @@ -0,0 +1,743 @@ +import AppKit +import Foundation +import LocalLMLabSDKCore +import LocalLMLabSDKInference +import LocalLMLabSDKRemote +import Observation +import OpenJevKit +import UniformTypeIdentifiers + +/// One backend's answer (or error) for one input. +struct BackendRun: Identifiable { + var id: String { label } + let label: String + var result: DecisionResult? + var error: String? + /// What the backend was, for the results CSV. + var info: ResultsCSV.BackendInfo? = nil + /// Hosted only, as the provider reported them. + var inputTokens: Int? = nil + var costUSD: Double? = nil +} + +/// One batch input with every backend's answers. +struct BatchRow: Identifiable { + let id = UUID() + let input: String + let runs: [BackendRun] +} + +/// A local model the SDK has downloaded and verified, as the picker shows it. +struct LocalChoice: Identifiable, Hashable { + var id: String { repoID } + let repoID: String + let sizeBytes: Int64? + let isMoE: Bool + + var displayName: String { repoID.split(separator: "/").last.map(String.init) ?? repoID } + var sizeText: String { sizeBytes.map { ByteCountFormatter.string(fromByteCount: $0, countStyle: .file) } ?? "?" } +} + +/// Everything runs on the SDK's decision API: hosted Jev through `lab.decide` +/// with `JevDecisionProvider`; local OpenJev through `OpenJevDecisionProvider` (with the editor's +/// wrapper, and `decideWithDiagnostics` for the prompt view); models listed, checked and +/// downloaded by `MLXModelProvider`. +@MainActor +@Observable +final class AppModel { + // Local models + var models: [LocalChoice] = [] + /// In the Hugging Face cache but not downloaded through the SDK, so not usable until verified. + var unverifiedRepos: [String] = [] + var selectedModelID: String? { didSet { persistSoon() } } + + // What the developer is editing + var set: QuestionSet = Presets.triage { didSet { persistSoon() } } + var input = Presets.triageInput { didSet { persistSoon() } } + var batchText = Presets.triageBatch { didSet { persistSoon() } } + var samples: [(String, String)] = [] { didSet { persistSoon() } } + + // Results: one run per backend, in backend order. + var results: [BackendRun] = [] + var batchRows: [BatchRow] = [] + var batchProgress: (done: Int, total: Int)? + /// Groups one batch's rows in the results CSV. + var batchRunID: String? + var batchStartedAt: Date? + /// Where "Append results to CSV" last wrote. + var resultsCSVPath: String? = UserDefaults.standard.string(forKey: "resultsCSVPath") { + didSet { UserDefaults.standard.set(resultsCSVPath, forKey: "resultsCSVPath") } + } + /// Rows appended by the last write, for the status line. + var lastAppend: (rows: Int, file: String)? + var isRunning = false + var errorMessage: String? + + // Backends + var useLocal = true { didSet { UserDefaults.standard.set(useLocal, forKey: "useLocal") } } + var hostedEnabled: [HostedBackend: Bool] = [:] { didSet { saveHostedSettings() } } + var hostedModel: [HostedBackend: String] = [:] { didSet { saveHostedSettings() } } + var hostedModelChoices: [HostedBackend: [String]] = [:] + + // Calibration (the developer's: fitted on answers they mark correct in the batch grid) + /// Input → question name → the correct answer's key. + var labels: [String: [String: String]] = [:] { didSet { persistSoon() } } + var calibration: FittedCalibration? { didSet { persistSoon() } } + /// Show local answers with `calibration` applied, as `OpenJevDecisionProvider(calibration:)` would. + var applyCalibration = false { didSet { persistSoon() } } + + // Download / verify + var downloadRepo = "" + var preflight: PreflightResult? + var downloadProgress: Double? + + /// The SDK front door hosted runs go through. + let lab = LocalLMLab() + /// Local models: listing, preflight, download, and the weights OpenJev scores on. + let mlx = MLXModelProvider() + private var batchTask: Task? + private var fileURL: URL? + + var selectedModel: LocalChoice? { models.first { $0.id == selectedModelID } } + + init() { + assert(QuestionSet.defaultSystem == OpenJevWrapper.default.system + && QuestionSet.defaultInputLabel == OpenJevWrapper.default.inputLabel, + "OpenJevKit's mirrored default wrapper drifted from the SDK's") + restoreWorkspace() + refreshModels() + let d = UserDefaults.standard + if d.object(forKey: "useLocal") != nil { useLocal = d.bool(forKey: "useLocal") } + for b in HostedBackend.allCases { + hostedEnabled[b] = d.bool(forKey: "remote.\(b.rawValue).on") + hostedModel[b] = d.string(forKey: "remote.\(b.rawValue).model") ?? b.defaultModel + hostedModelChoices[b] = b.suggestedModels + } + } + + func refreshModels() { + let probe = OpenJevDecisionProvider(mlx: mlx) + models = mlx.installed.map { m in + LocalChoice(repoID: m.repoID, sizeBytes: m.sizeBytes, + isMoE: probe.warnings(for: Self.openJevID(m.repoID)).contains(.mixtureOfExperts)) + }.sorted { ($0.sizeBytes ?? 0) < ($1.sizeBytes ?? 0) } + let verified = Set(models.map(\.repoID)) + unverifiedRepos = ModelCatalog.cachedRepoIDs().filter { !verified.contains($0) } + if selectedModelID == nil || selectedModel == nil { + // Default to the smallest non-MoE model of at least ~2 GB (Qwen3-4B-class, the eval's + // pick), else the smallest non-MoE one. + let dense = models.filter { !$0.isMoE } + selectedModelID = (dense.first { ($0.sizeBytes ?? 0) >= 2_000_000_000 } ?? dense.first ?? models.first)?.id + } + } + + static func openJevID(_ repo: String) -> ModelID { ModelID(scheme: "openjev", rest: repo)! } + + // MARK: Backends + + private func saveHostedSettings() { + let d = UserDefaults.standard + for b in HostedBackend.allCases { + d.set(hostedEnabled[b] ?? false, forKey: "remote.\(b.rawValue).on") + d.set(hostedModel[b], forKey: "remote.\(b.rawValue).model") + } + } + + func apiKey(_ b: HostedBackend) -> String? { Keychain.get("key.\(b.rawValue)") } + func setAPIKey(_ key: String?, _ b: HostedBackend) { Keychain.set(key, for: "key.\(b.rawValue)") } + + var enabledHosted: [HostedBackend] { HostedBackend.allCases.filter { hostedEnabled[$0] == true } } + + var localLabel: String { "Local · \(selectedModel?.displayName ?? "no model")" } + + func hostedLabel(_ b: HostedBackend) -> String { b.label(model: hostedModel[b] ?? b.defaultModel) } + + /// Labels of the backends a run will use, in display order. + var activeBackendLabels: [String] { + (useLocal ? [localLabel] : []) + enabledHosted.map(hostedLabel) + } + + func refreshHostedModels(_ b: HostedBackend) async { + hostedModelChoices[b] = await b.listModels() + } + + private struct HostedPlan { + let route: RouteName + let label: String + let requestPreview: String + let info: ResultsCSV.BackendInfo + } + + /// Register the enabled hosted backends and point a decision route at each. Re-registered every + /// run so a changed key or model takes effect. + private func hostedPlans(for request: DecisionRequest) -> [HostedPlan] { + enabledHosted.compactMap { b in + let model = hostedModel[b] ?? b.defaultModel + guard let id = b.modelID(model) else { return nil } + lab.models.replace(decision: JevDecisionProvider(b.config(key: b.needsKey ? apiKey(b) : nil))) + let route = RouteName("hosted.\(b.rawValue)") + lab.models.route(decision: route, to: id) + let body = (try? JevDecisionProvider.requestBody(model: model, request: request)) + .flatMap { try? JSONSerialization.jsonObject(with: $0) } + .flatMap { try? JSONSerialization.data(withJSONObject: $0, options: [.prettyPrinted]) } + .map { String(decoding: $0, as: UTF8.self) } ?? "" + return HostedPlan(route: route, label: hostedLabel(b), + requestPreview: "POST \(b.config(key: nil).endpoint.absoluteString)\n\(body)", + info: .init(backend: b.rawValue, model: model, revision: nil, sizeBytes: nil, isMoE: nil, + wrapperLabel: nil, wrapperSystem: nil)) + } + } + + /// Run `input` on every enabled backend, concurrently. + private func runAll(_ input: String) async -> [BackendRun] { + let questions = set.questions + let request = DecisionRequest(state: .text(input), questions: questions.map(\.sdkQuestion)) + let plans = hostedPlans(for: request) + let local: (LocalChoice, OpenJevDecisionProvider)? = useLocal ? selectedModel.map { m in + (m, OpenJevDecisionProvider(mlx: mlx, wrapper: OpenJevWrapper(system: set.system, inputLabel: set.inputLabel))) + } : nil + let localLabel = self.localLabel + let localInfo = local.map { m, openjev in + ResultsCSV.BackendInfo(backend: "local", model: m.repoID, + revision: mlx.installed.first { $0.repoID == m.repoID }?.resolvedRevision, + sizeBytes: m.sizeBytes, isMoE: m.isMoE, + wrapperLabel: openjev.wrapper.inputLabel, wrapperSystem: openjev.wrapper.system) + } + + return await withTaskGroup(of: (Int, BackendRun).self) { group in + if let (m, openjev) = local { + // Local runs call the provider directly for its diagnostics (exact prompts, label + // mass), validating against its limits first as lab.decide would. + group.addTask { + do { + try request.validate(against: openjev.limits) + let (d, diag) = try await openjev.decideWithDiagnostics(request, using: Self.openJevID(m.repoID)) + let qd = diag.questions.mapValues { + QuestionDiagnostics(labels: $0.labels, prompt: $0.prompt, labelMass: $0.labelMass, + milliseconds: Self.ms($0.latency)) + } + let rows = QuestionResult.rows(for: questions, decision: d, diagnostics: qd) + return (0, BackendRun(label: localLabel, result: DecisionResult( + input: input, results: rows, sharedPrefixTokens: diag.sharedPrefixTokens, + milliseconds: Self.ms(d.latency), backend: localLabel), info: localInfo)) + } catch { + return (0, BackendRun(label: localLabel, error: error.localizedDescription, info: localInfo)) + } + } + } + for (i, plan) in plans.enumerated() { + group.addTask { @MainActor in + do { + let d = try await self.lab.decide(route: plan.route, state: .text(input), questions: request.questions) + let rows = QuestionResult.rows(for: questions, decision: d, prompt: plan.requestPreview) + return (i + 1, BackendRun(label: plan.label, result: DecisionResult( + input: input, results: rows, sharedPrefixTokens: 0, milliseconds: Self.ms(d.latency), + backend: plan.label, usage: d.usage?.summary), info: plan.info, + inputTokens: d.usage?.inputTokens, costUSD: d.usage?.cost)) + } catch { + return (i + 1, BackendRun(label: plan.label, error: error.localizedDescription, info: plan.info)) + } + } + } + var out: [(Int, BackendRun)] = [] + for await r in group { out.append(r) } + return out.sorted { $0.0 < $1.0 }.map(\.1) + } + } + + nonisolated private static func ms(_ d: Duration) -> Double { + Double(d.components.seconds) * 1000 + Double(d.components.attoseconds) / 1e15 + } + + // MARK: Running + + private func validationProblem() -> String? { + if activeBackendLabels.isEmpty { return "Turn on at least one backend." } + if useLocal && selectedModel == nil { return "Pick a local model (Models in the toolbar), or turn Local off in Backends." } + if set.questions.isEmpty { return "Add at least one question." } + if let q = set.questions.first(where: { !$0.problems.isEmpty }) { + return "“\(q.name)”: \(q.problems.joined(separator: "; "))." + } + return nil + } + + func run() async { + guard !isRunning else { return } + if let problem = validationProblem() { errorMessage = problem; return } + errorMessage = nil + isRunning = true + defer { isRunning = false } + results = await runAll(input) + } + + var batchInputs: [String] { + batchText.split(separator: "\n").map { $0.trimmingCharacters(in: .whitespaces) }.filter { !$0.isEmpty } + } + + func runBatch() { + guard !isRunning else { return } + if let problem = validationProblem() { errorMessage = problem; return } + errorMessage = nil + let inputs = batchInputs + let pause = enabledHosted.map(\.minInterval).max() ?? .zero + batchRows = [] + let started = Date() + batchRunID = ResultsCSV.newRunID(at: started) + batchStartedAt = started + lastAppend = nil + batchTask = Task { + isRunning = true + defer { isRunning = false; batchProgress = nil } + batchProgress = (0, inputs.count) + for (i, line) in inputs.enumerated() { + if Task.isCancelled { break } + batchRows.append(BatchRow(input: line, runs: await runAll(line))) + batchProgress = (i + 1, inputs.count) + if pause > .zero { try? await Task.sleep(for: pause) } + } + } + } + + func stopBatch() { batchTask?.cancel() } + + // MARK: Calibration + + func label(input: String, question: String) -> String? { labels[input]?[question] } + + func setLabel(_ key: String?, input: String, question: String) { + var row = labels[input] ?? [:] + row[question] = key + labels[input] = row.isEmpty ? nil : row + } + + var markedCount: Int { labels.values.map(\.count).reduce(0, +) } + + var calibrationSamples: [DecisionCalibration.Sample] { Calibrator.samples(rows: batchRows, labels: labels) } + + var selectedRevision: String? { + mlx.installed.first { $0.repoID == selectedModel?.repoID }?.resolvedRevision + } + + /// Why the current calibration may not hold for the current model and wrapper; nil if it does. + var calibrationMismatch: String? { + calibration?.mismatch(repoID: selectedModel?.repoID, revision: selectedRevision, + system: set.system, inputLabel: set.inputLabel) + } + + /// Fit on the marked answers with the SDK's `DecisionCalibration.fit`. + func fitCalibration() { + guard let m = selectedModel else { errorMessage = "Pick a local model first."; return } + let samples = calibrationSamples + guard !samples.isEmpty else { errorMessage = "Mark some correct answers in the batch grid first."; return } + let fitted = DecisionCalibration.fit(samples) + calibration = FittedCalibration( + noulTemperature: fitted.noulTemperature, choiceTemperature: fitted.choiceTemperature, + scoreTemperature: fitted.scoreTemperature, repoID: m.repoID, revision: selectedRevision ?? "?", + system: set.system, inputLabel: set.inputLabel, fittedAt: Date(), + samples: Dictionary(grouping: samples, by: \.kind.rawValue).mapValues(\.count)) + applyCalibration = true + } + + /// `run` as JevDK shows it: local answers calibrated when a matching calibration is applied. + func display(_ run: BackendRun) -> BackendRun { + guard applyCalibration, let c = calibration, calibrationMismatch == nil, + run.label.hasPrefix("Local"), let r = run.result else { return run } + return BackendRun(label: run.label, result: Calibrator.applying(c.sdk, to: r), error: run.error, + info: run.info, inputTokens: run.inputTokens, costUSD: run.costUSD) + } + + /// Per backend: answers matching the marked correct answer, out of those marked. + var markedAccuracy: [(label: String, right: Int, total: Int)] { + var tally: [String: (Int, Int)] = [:] + var order: [String] = [] + for row in batchRows { + guard let marks = labels[row.input] else { continue } + for run in row.runs { + guard let r = run.result else { continue } + if tally[run.label] == nil { order.append(run.label); tally[run.label] = (0, 0) } + for q in r.results { + guard let key = marks[q.question.name] else { continue } + tally[run.label]!.1 += 1 + if q.answer == key { tally[run.label]!.0 += 1 } + } + } + } + return order.map { ($0, tally[$0]!.0, tally[$0]!.1) } + } + + // MARK: Question set for the app, answer sets + + /// What the questions are being tested with, for the exported question set: the local model, + /// its revision, the wrapper and (when it matches) the calibration. Nil with Local off. + var exportTuning: DecisionQuestionSet.Tuning? { + guard useLocal, let m = selectedModel else { return nil } + let cal = calibrationMismatch == nil ? calibration?.sdk : nil + return .init(model: Self.openJevID(m.repoID), revision: selectedRevision, system: set.system, + inputLabel: set.inputLabel, calibration: cal, testedAt: Date()) + } + + /// The questions as the SDK's `DecisionQuestionSet` JSON, for the app to bundle and load, so + /// the shipped questions are exactly the tested ones. + func exportForApp() { + if let q = set.questions.first(where: { !$0.problems.isEmpty }) { + errorMessage = "Fix “\(q.name)” first: \(q.problems.joined(separator: "; "))." + return + } + let panel = NSSavePanel() + panel.allowedContentTypes = [.json] + let base = set.name.isEmpty ? "questions" : set.name.lowercased().replacingOccurrences(of: " ", with: "-") + panel.nameFieldStringValue = "\(base).decisions.json" + panel.message = "A question set for your app: bundle it and load it with DecisionQuestionSet(contentsOf:)." + guard panel.runModal() == .OK, let url = panel.url else { return } + do { + let file = DecisionQuestionSet(name: set.name, questions: set.questions.map(\.sdkQuestion), tuning: exportTuning) + try file.jsonData().write(to: url) + errorMessage = nil + } catch { + errorMessage = "Couldn't export: \(error.localizedDescription)" + } + } + + /// Load an answer set (CSV: `input`, then a column per question with the correct answer) into + /// the batch list and the marks. Inputs with line breaks are joined onto one line, since the + /// batch list is one input per line. + func importAnswerSet() { + let panel = NSOpenPanel() + panel.allowedContentTypes = [.commaSeparatedText, .plainText] + panel.message = "An answer set: a CSV with an input column and one column per question, holding the correct answer." + guard panel.runModal() == .OK, let url = panel.url else { return } + do { + let answers = try DecisionAnswerSet(csv: String(contentsOf: url, encoding: .utf8)) + let questions = set.questions.map(\.sdkQuestion) + let problems = answers.problems(for: questions) + guard problems.isEmpty else { + errorMessage = "\(url.lastPathComponent) doesn't fit these questions: " + problems.prefix(3).joined(separator: " ") + + (problems.count > 3 ? " (and \(problems.count - 3) more)" : "") + return + } + var newLabels: [String: [String: String]] = [:] + var inputs: [String] = [] + var joined = 0 + for e in answers.examples { + let one = e.input.split(whereSeparator: \.isNewline).joined(separator: " ") + if one != e.input { joined += 1 } + inputs.append(one) + var marks: [String: String] = [:] + for q in questions { + if let raw = e.answers[q.id], let key = DecisionAnswerSet.answerKey(raw, for: q) { marks[q.id] = key } + } + if !marks.isEmpty { newLabels[one] = marks } + } + batchText = inputs.joined(separator: "\n") + labels = newLabels + batchRows = [] + errorMessage = joined > 0 ? "Imported \(inputs.count) inputs; \(joined) had line breaks and were joined onto one line." : nil + } catch { + errorMessage = "Couldn't import: \(error.localizedDescription)" + } + } + + /// The batch inputs and their marks as an answer-set CSV. + func exportAnswerSet() { + let panel = NSSavePanel() + panel.allowedContentTypes = [.commaSeparatedText] + panel.nameFieldStringValue = "\(set.name.isEmpty ? "answers" : set.name.lowercased().replacingOccurrences(of: " ", with: "-")).answers.csv" + guard panel.runModal() == .OK, let url = panel.url else { return } + let answers = DecisionAnswerSet(examples: batchInputs.map { .init(input: $0, answers: labels[$0] ?? [:]) }) + do { + try answers.csv(questionIDs: set.questions.map(\.name)).write(to: url, atomically: true, encoding: .utf8) + errorMessage = nil + } catch { + errorMessage = "Couldn't export: \(error.localizedDescription)" + } + } + + // MARK: Results CSV + + /// Append the last batch's results (as shown: calibrated if a calibration is applied) to a + /// results CSV, one row per input × question × backend. `choose` asks for a file; otherwise the + /// last one is used. + func appendResults(choose: Bool) { + guard !batchRows.isEmpty, let runID = batchRunID else { errorMessage = "Run a batch first."; return } + var url = resultsCSVPath.map { URL(fileURLWithPath: $0) } + if choose || url == nil { + let panel = NSSavePanel() + panel.allowedContentTypes = [.commaSeparatedText] + panel.nameFieldStringValue = url?.lastPathComponent ?? "jevdk-results.csv" + panel.message = "Choose a results CSV to append to, or name a new one." + if let dir = url?.deletingLastPathComponent() { panel.directoryURL = dir } + guard panel.runModal() == .OK, let picked = panel.url else { return } + url = picked + } + guard let url else { return } + let run = ResultsCSV.RunInfo(runID: runID, savedAt: batchStartedAt ?? Date(), sdkVersion: LocalLMLabSDKVersion.current, + machine: ResultsCSV.machine, questionSet: set.name) + var lines: [String] = [] + for row in batchRows { + for r in row.runs.map(display) { + guard let result = r.result, let info = r.info else { continue } + lines += ResultsCSV.rows(run: run, backend: info, result: result, labels: labels[row.input] ?? [:], + inputTokens: r.inputTokens, costUSD: r.costUSD) + } + } + do { + try ResultsCSV.append(lines, to: url) + resultsCSVPath = url.path + lastAppend = (lines.count, url.lastPathComponent) + errorMessage = nil + } catch { + errorMessage = error.localizedDescription + } + } + + // MARK: Models + + /// Text models tried as deciders or worth trying; all MLX 4-bit builds on Hugging Face. + static let recommended: [(repo: String, note: String)] = [ + ("mlx-community/Qwen3-4B-4bit", "2.3 GB · tested as a decider: the current pick"), + ("mlx-community/Qwen3-1.7B-4bit", "1.0 GB · tested: fast, but weaker on yes/no and answer checks"), + ("mlx-community/Qwen3-8B-4bit", "4.6 GB · untested here; a bigger Qwen3"), + ("mlx-community/Llama-3.2-3B-Instruct-4bit", "1.8 GB · untested here"), + ("mlx-community/Phi-4-mini-instruct-4bit", "2.2 GB · untested here"), + ("mlx-community/Qwen2.5-3B-Instruct-4bit", "1.8 GB · untested here"), + ] + + struct HubModel: Identifiable, Decodable { + let id: String + let downloads: Int? + let pipeline_tag: String? + } + + /// MLX models on Hugging Face matching `query`, most downloaded first. + func searchHub(_ query: String) async -> [HubModel] { + var c = URLComponents(string: "https://huggingface.co/api/models")! + c.queryItems = [.init(name: "filter", value: "mlx"), .init(name: "search", value: query), + .init(name: "sort", value: "downloads"), .init(name: "limit", value: "25")] + guard let url = c.url, let (data, _) = try? await URLSession.shared.data(from: url), + let models = try? JSONDecoder().decode([HubModel].self, from: data) else { + errorMessage = "Couldn't search Hugging Face." + return [] + } + return models + } + + func remove(_ repo: String) { + do { + try mlx.remove(ModelID(scheme: "mlx", rest: repo)!) + if selectedModelID == repo { selectedModelID = nil } + refreshModels() + } catch { + errorMessage = "Couldn't remove \(repo): \(error.localizedDescription)" + } + } + + // MARK: Download / verify (through the SDK) + + /// The SDK's no-download checks: repo reachable, MLX format, supported architecture, size vs + /// this Mac's memory and disk. + func checkRepo() async { + preflight = nil + let repo = downloadRepo.trimmingCharacters(in: .whitespaces) + guard repo.contains("/") else { return } + do { + preflight = try await mlx.validate(repo) + } catch { + errorMessage = "Couldn't check \(repo): \(error.localizedDescription)" + } + } + + /// Download `repo` (or `downloadRepo`) through the SDK, which verifies the weights. Files + /// already in the cache are verified rather than fetched again. + func download(_ repo: String? = nil) async { + let repo = (repo ?? downloadRepo).trimmingCharacters(in: .whitespaces) + guard repo.contains("/") else { errorMessage = "Enter a repo id like mlx-community/Qwen3-4B-4bit."; return } + errorMessage = nil + downloadProgress = 0 + defer { downloadProgress = nil } + do { + for try await event in mlx.download(repo) { + if case .progress(_, _, let f) = event { downloadProgress = f } + } + refreshModels() + selectedModelID = repo + downloadRepo = "" + preflight = nil + } catch { + errorMessage = "Download failed: \(error.localizedDescription)" + } + } + + // MARK: Question editing + + func addQuestion(_ kind: QuestionKind) { + let n = set.questions.count + 1 + switch kind { + case .noul: + set.questions.append(EditableQuestion(name: "question\(n)", kind: .noul, text: "")) + case .choice: + set.questions.append(EditableQuestion(name: "question\(n)", kind: .choice, text: "", options: [ + AnswerOption(key: "first", description: ""), AnswerOption(key: "second", description: ""), + ])) + case .score: + set.questions.append(EditableQuestion(name: "question\(n)", kind: .score, text: "", options: [ + AnswerOption(key: "0", description: "low"), AnswerOption(key: "1", description: "medium"), + AnswerOption(key: "2", description: "high"), + ])) + } + } + + func duplicate(_ q: EditableQuestion) { + guard let i = set.questions.firstIndex(where: { $0.id == q.id }) else { return } + var copy = q + copy.id = UUID() + copy.name = q.name + "2" + copy.options = q.options.map { AnswerOption(key: $0.key, description: $0.description) } + set.questions.insert(copy, at: i + 1) + } + + func remove(_ q: EditableQuestion) { set.questions.removeAll { $0.id == q.id } } + + func move(_ q: EditableQuestion, by offset: Int) { + guard let i = set.questions.firstIndex(where: { $0.id == q.id }) else { return } + let j = i + offset + guard set.questions.indices.contains(j) else { return } + set.questions.swapAt(i, j) + } + + // MARK: Workspace (implicit save and restore) + + /// What JevDK brings back on the next launch, like Prompt Playground keeps its last prompt: the + /// questions and wrapper, the inputs, marks, calibration and the selected model. Results aren't + /// kept; run again. Saved to UserDefaults shortly after each change. + struct Workspace: Codable { + var set: QuestionSet + var input: String + var batch: String + var samples: [[String]] + var labels: [String: [String: String]] + var calibration: FittedCalibration? + var applyCalibration: Bool + var selectedModelID: String? + } + + private static let workspaceKey = "jevdk.workspace" + private var persistTask: Task? + private var restoring = false + + private func persistSoon() { + guard !restoring else { return } + persistTask?.cancel() + persistTask = Task { [weak self] in + try? await Task.sleep(for: .milliseconds(400)) + guard !Task.isCancelled, let self else { return } + self.persistNow() + } + } + + func persistNow() { + let w = Workspace(set: set, input: input, batch: batchText, samples: samples.map { [$0.0, $0.1] }, + labels: labels, calibration: calibration, applyCalibration: applyCalibration, + selectedModelID: selectedModelID) + if let data = try? JSONEncoder().encode(w) { UserDefaults.standard.set(data, forKey: Self.workspaceKey) } + } + + private func restoreWorkspace() { + guard let data = UserDefaults.standard.data(forKey: Self.workspaceKey), + let w = try? JSONDecoder().decode(Workspace.self, from: data) else { + samples = Presets.all.first { $0.set.name == set.name }?.samples ?? [] + return + } + restoring = true + defer { restoring = false } + set = w.set + input = w.input + batchText = w.batch + samples = w.samples.compactMap { $0.count == 2 ? ($0[0], $0[1]) : nil } + labels = w.labels + calibration = w.calibration + applyCalibration = w.applyCalibration + selectedModelID = w.selectedModelID // refreshModels keeps it if still installed + } + + /// Nothing to lose by replacing the workspace: it's an unedited example with no marks or + /// calibration. + var workspaceIsPristine: Bool { + labels.isEmpty && calibration == nil + && Presets.all.contains { $0.set == set && $0.batch == batchText } + } + + func loadPreset(_ p: Presets.Preset) { + set = p.set + input = p.input + batchText = p.batch + samples = p.samples + labels = [:] + calibration = nil + applyCalibration = false + results = [] + batchRows = [] + fileURL = nil + } + + // MARK: Files + + /// Save the question set, inputs, marks and calibration to a JSON file. Returns whether it saved + /// (false if the panel was cancelled or the write failed). + @discardableResult + func save(as: Bool = false) -> Bool { + var url = fileURL + if url == nil || `as` { + let panel = NSSavePanel() + panel.allowedContentTypes = [.json] + panel.nameFieldStringValue = set.name.isEmpty ? "questions.json" : "\(set.name).json" + guard panel.runModal() == .OK, let picked = panel.url else { return false } + url = picked + } + guard let url else { return false } + do { + let enc = JSONEncoder() + enc.outputFormatting = [.prettyPrinted, .sortedKeys] + try enc.encode(SavedSet(set: set, input: input, batch: batchText, labels: labels, calibration: calibration)).write(to: url) + fileURL = url + return true + } catch { + errorMessage = "Couldn't save: \(error.localizedDescription)" + return false + } + } + + func open() { + let panel = NSOpenPanel() + panel.allowedContentTypes = [.json] + guard panel.runModal() == .OK, let url = panel.url else { return } + do { + let data = try Data(contentsOf: url) + if let saved = try? JSONDecoder().decode(SavedSet.self, from: data) { + set = saved.set + input = saved.input ?? input + batchText = saved.batch ?? batchText + labels = saved.labels ?? [:] + calibration = saved.calibration + applyCalibration = saved.calibration != nil + } else { + set = try JSONDecoder().decode(QuestionSet.self, from: data) + } + results = [] + batchRows = [] + samples = [] + fileURL = url + } catch { + errorMessage = "Couldn't open \(url.lastPathComponent): \(error.localizedDescription)" + } + } +} + +/// What JevDK writes to disk: the question set plus the inputs being tried. +struct SavedSet: Codable { + var set: QuestionSet + var input: String? + var batch: String? + /// Input → question name → the correct answer's key, as marked in the batch grid. + var labels: [String: [String: String]]? + var calibration: FittedCalibration? +} diff --git a/examples/jevdk/Sources/JevDK/Calibration.swift b/examples/jevdk/Sources/JevDK/Calibration.swift new file mode 100644 index 0000000..45ca080 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/Calibration.swift @@ -0,0 +1,123 @@ +import Foundation +import LocalLMLabSDKCore +import LocalLMLabSDKInference +import OpenJevKit + +/// A calibration the developer fitted in JevDK, with what it's valid for. Calibration is the app's: +/// it holds for this model revision, wrapper and kind of question only, so JevDK records all three +/// and flags the calibration when any changes. +struct FittedCalibration: Codable, Hashable { + var noulTemperature: Double + var choiceTemperature: Double + var scoreTemperature: Double + /// What it was fitted for. + var repoID: String + var revision: String + var system: String + var inputLabel: String + var fittedAt: Date + /// Marked answers per kind it was fitted on. + var samples: [String: Int] + + var sdk: DecisionCalibration { + DecisionCalibration(noulTemperature: noulTemperature, choiceTemperature: choiceTemperature, scoreTemperature: scoreTemperature) + } + + /// Why it may not hold for the current setup, or nil if it matches. + func mismatch(repoID: String?, revision: String?, system: String, inputLabel: String) -> String? { + var reasons: [String] = [] + if repoID != self.repoID { reasons.append("it was fitted on \(self.repoID)") } + else if let revision, revision != self.revision { reasons.append("the model has been updated since") } + if system != self.system || inputLabel != self.inputLabel { reasons.append("the system instructions changed since") } + return reasons.isEmpty ? nil : reasons.joined(separator: ", ") + } + + /// What the developer pastes into their app. The questions, wrapper and this calibration travel + /// in the exported question set (File → Export for App…), so the code only loads it. + var swiftSnippet: String { + """ + // Export the question set from JevDK (File → Export for App…) and bundle it with your app. + // It holds the questions, the wrapper and this calibration, fitted on \(samples.values.reduce(0, +)) marked answers + // for \(repoID) @ \(revision.prefix(12)) (\(fittedAt.formatted(date: .abbreviated, time: .omitted))). + let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "questions.decisions", withExtension: "json")!) + let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) + try lab.models.register(decision: openjev) + lab.models.route(decision: "decide", to: ModelID("openjev:\(repoID)")!) + let d = try await lab.decide(route: "decide", state: .text(input), questions: set.questions) + + // Or, without the file: + // OpenJevDecisionProvider(mlx: mlx, + // wrapper: OpenJevWrapper(system: \(Self.literal(system)), inputLabel: \(Self.literal(inputLabel))), + // calibration: DecisionCalibration(noulTemperature: \(noulTemperature), choiceTemperature: \(choiceTemperature), scoreTemperature: \(scoreTemperature))) + """ + } + + private static func literal(_ s: String) -> String { + "\"" + s.replacingOccurrences(of: "\\", with: "\\\\").replacingOccurrences(of: "\"", with: "\\\"") + "\"" + } +} + +/// Before → after on the marked answers, per kind. +struct CalibrationReport { + struct Row: Identifiable { + var id: String { kind.rawValue } + let kind: DecisionCalibration.Sample.Kind + let temperature: Double + let before: DecisionCalibration.Measurement + let after: DecisionCalibration.Measurement + } + let rows: [Row] +} + +extension DecisionCalibration.Sample.Kind { + init(_ k: QuestionKind) { + switch k { + case .noul: self = .noul + case .choice: self = .choice + case .score: self = .score + } + } + + var displayName: String { + switch self { + case .noul: "Noul (yes / no)" + case .choice: "Choice" + case .score: "Score" + } + } +} + +enum Calibrator { + /// Calibration samples from the local backend's raw answers in `rows`, for every question whose + /// correct answer the developer marked. + static func samples(rows: [BatchRow], labels: [String: [String: String]], localLabelPrefix: String = "Local") + -> [DecisionCalibration.Sample] + { + rows.flatMap { row -> [DecisionCalibration.Sample] in + guard let marks = labels[row.input], + let local = row.runs.first(where: { $0.label.hasPrefix(localLabelPrefix) })?.result else { return [] } + return local.results.compactMap { r in + guard let key = marks[r.question.name], let i = r.keys.firstIndex(of: key) else { return nil } + return .init(kind: .init(r.question.kind), probabilities: r.rawProbabilities, correct: [i]) + } + } + } + + static func report(_ samples: [DecisionCalibration.Sample], _ calibration: DecisionCalibration) -> CalibrationReport { + CalibrationReport(rows: DecisionCalibration.Sample.Kind.allCases.compactMap { k in + let s = samples.filter { $0.kind == k } + guard !s.isEmpty else { return nil } + return .init(kind: k, temperature: calibration.temperature(for: k), + before: DecisionCalibration.none.measure(s), after: calibration.measure(s)) + }) + } + + /// `result`'s local answers as `calibration` reports them. + static func applying(_ calibration: DecisionCalibration, to result: DecisionResult) -> DecisionResult { + let rows = result.results.map { r in + r.showing(calibration.apply(r.rawProbabilities, kind: .init(r.question.kind)), calibrated: true) + } + return DecisionResult(input: result.input, results: rows, sharedPrefixTokens: result.sharedPrefixTokens, + milliseconds: result.milliseconds, backend: result.backend, usage: result.usage) + } +} diff --git a/examples/jevdk/Sources/JevDK/CalibrationSheet.swift b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift new file mode 100644 index 0000000..9593568 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift @@ -0,0 +1,139 @@ +import AppKit +import LocalLMLabSDKCore +import LocalLMLabSDKInference +import OpenJevKit +import SwiftUI + +/// Fit the local model's confidence to the answers the developer marked correct, see the +/// before → after, preview it, and copy the code for the app. +struct CalibrationSheet: View { + @Bindable var model: AppModel + @Environment(\.dismiss) private var dismiss + + var body: some View { + let samples = model.calibrationSamples + let byKind = Dictionary(grouping: samples, by: \.kind) + VStack(alignment: .leading, spacing: 14) { + Text("Calibrate the local model").font(.jHeadline) + Text(""" + A small local model often says 100% even when it's wrong. Calibration softens its \ + confidence so that "80% sure" means right about 80% of the time. It never changes which \ + answer wins. It's fitted on answers you mark correct in the batch grid, for this model, \ + these system instructions and your kind of questions, so your app uses the result you \ + fit here. + """) + .font(.jCallout).foregroundStyle(.secondary) + .fixedSize(horizontal: false, vertical: true) + + GroupBox("Your marked answers") { + VStack(alignment: .leading, spacing: 6) { + if samples.isEmpty { + Text("None yet. Run a batch with the local backend on, then use “mark…” under each cell to record the correct answer.") + .font(.jCallout).foregroundStyle(.secondary) + } else { + HStack(spacing: 16) { + ForEach(DecisionCalibration.Sample.Kind.allCases, id: \.self) { k in + kindCount(k, byKind[k]?.count ?? 0) + } + } + .font(.jCallout) + if (byKind.values.map(\.count).min() ?? 0) < 30 { + Text("Aim for 30 or more marked answers per kind you use; with fewer, the fit follows these examples too closely to carry over.") + .font(.jCaption).foregroundStyle(.orange) + } + } + let acc = model.markedAccuracy + if !acc.isEmpty { + Divider() + Text("Right answers on your marks").font(.jCaption.weight(.semibold)) + ForEach(acc, id: \.label) { a in + Text("\(a.label): \(a.right)/\(a.total) (\(percent(Double(a.right) / Double(max(a.total, 1)))))") + .font(.jCaption.monospacedDigit()) + } + } + } + .frame(maxWidth: .infinity, alignment: .leading) + .padding(4) + } + + HStack { + Button("Fit calibration") { model.fitCalibration() } + .buttonStyle(.borderedProminent) + .disabled(samples.isEmpty || model.selectedModel == nil) + if let m = model.selectedModel { + Text("for \(m.repoID)").font(.jCaption).foregroundStyle(.secondary) + } + } + + if let c = model.calibration { + result(c, samples: samples) + } + + HStack { + Spacer() + Button("Done") { dismiss() }.keyboardShortcut(.defaultAction) + } + } + .padding(20) + .frame(width: 680) + .font(.jBody) + } + + private func kindCount(_ k: DecisionCalibration.Sample.Kind, _ n: Int) -> some View { + let color: Color = n == 0 ? .secondary : (n >= 30 ? .primary : .orange) + return Label("\(k.displayName): \(n)", systemImage: n >= 30 ? "checkmark.circle" : "circle.dotted") + .foregroundStyle(color) + } + + @ViewBuilder + private func result(_ c: FittedCalibration, samples: [DecisionCalibration.Sample]) -> some View { + GroupBox("Fitted") { + VStack(alignment: .leading, spacing: 8) { + if let why = model.calibrationMismatch { + Label("Doesn't match the current setup: \(why). Refit, or switch back.", systemImage: "exclamationmark.triangle") + .font(.jCallout).foregroundStyle(.orange) + } + let report = Calibrator.report(samples, c.sdk) + Grid(alignment: .leading, horizontalSpacing: 14, verticalSpacing: 4) { + GridRow { + Text("Kind"); Text("Temperature"); Text("Marked"); Text("Calibration error"); Text("Log-loss") + } + .font(.jCaption.weight(.semibold)) + ForEach(report.rows) { r in + GridRow { + Text(r.kind.displayName) + Text(String(format: "%.2f", r.temperature)) + Text("\(r.before.count)") + Text(String(format: "%.3f → %.3f", r.before.expectedCalibrationError, r.after.expectedCalibrationError)) + Text(String(format: "%.2f → %.2f", r.before.logLoss, r.after.logLoss)) + } + .font(.jCaption.monospacedDigit()) + } + } + Text("Lower is better. Measured on the same answers it was fitted on, so expect a little less on new inputs. Accuracy doesn't change.") + .font(.jCaption).foregroundStyle(.secondary) + + Toggle("Show local results calibrated in JevDK", isOn: $model.applyCalibration) + .disabled(model.calibrationMismatch != nil) + + Text("For your app").font(.jCaption.weight(.semibold)) + Text(c.swiftSnippet) + .font(.system(size: 12, design: .monospaced)) + .textSelection(.enabled) + .padding(8) + .frame(maxWidth: .infinity, alignment: .leading) + .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 6)) + HStack { + Button("Export for App…") { model.exportForApp() } + Button("Copy code") { + NSPasteboard.general.clearContents() + NSPasteboard.general.setString(c.swiftSnippet, forType: .string) + } + Text("Saved with the question set (⌘S).").font(.jCaption).foregroundStyle(.secondary) + } + } + .frame(maxWidth: .infinity, alignment: .leading) + .padding(4) + } + } +} diff --git a/examples/jevdk/Sources/JevDK/EditorView.swift b/examples/jevdk/Sources/JevDK/EditorView.swift new file mode 100644 index 0000000..4853a9c --- /dev/null +++ b/examples/jevdk/Sources/JevDK/EditorView.swift @@ -0,0 +1,156 @@ +import OpenJevKit +import SwiftUI + +/// Left pane: the system instructions and the questions. +struct EditorView: View { + @Bindable var model: AppModel + @State private var showSystem = false + + var body: some View { + ScrollView { + VStack(alignment: .leading, spacing: 14) { + TextField("Question set name", text: $model.set.name) + .textFieldStyle(.plain) + .font(.jTitle3.weight(.semibold)) + + DisclosureGroup(isExpanded: $showSystem) { + TextEditor(text: $model.set.system) + .font(.system(size: 14, design: .monospaced)) + .frame(minHeight: 90) + .scrollContentBackground(.hidden) + .padding(6) + .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 6)) + Text("Local only: hosted Jev (Featherless, TypeSafe) never sees these instructions or the input's name, just the input and each question's text. Put anything a question needs into the question itself.") + .font(.jCaption) + .foregroundStyle(.secondary) + HStack { + Text("Input is called").font(.jCallout) + TextField("Input", text: $model.set.inputLabel) + .frame(width: 120) + Text("in the prompt").font(.jCallout).foregroundStyle(.secondary) + } + } label: { + Label("System instructions", systemImage: "text.alignleft") + .font(.jCallout.weight(.medium)) + } + + ForEach($model.set.questions) { $q in + QuestionCard(question: $q, model: model) + } + + Menu { + ForEach(QuestionKind.allCases, id: \.self) { kind in + Button(kind.displayName) { model.addQuestion(kind) } + } + } label: { + Label("Add question", systemImage: "plus") + } + .menuStyle(.borderlessButton) + .fixedSize() + + Text(""" + Tips from the eval: yes/no and multiple choice beat scales on small models; give \ + options short descriptions; only ask what a model can judge from the input and \ + stable general knowledge, never current facts. + """) + .font(.jCaption) + .foregroundStyle(.secondary) + } + .padding() + } + } +} + +struct QuestionCard: View { + @Binding var question: EditableQuestion + let model: AppModel + + var body: some View { + GroupBox { + VStack(alignment: .leading, spacing: 8) { + HStack { + TextField("name", text: $question.name) + .font(.system(size: 14, design: .monospaced).weight(.semibold)) + .textFieldStyle(.plain) + Spacer() + Picker("", selection: $question.kind) { + ForEach(QuestionKind.allCases, id: \.self) { Text($0.displayName).tag($0) } + } + .labelsHidden() + .fixedSize() + .onChange(of: question.kind) { _, kind in fillOptionsIfNeeded(kind) } + Menu { + Button("Duplicate (to compare wordings)") { model.duplicate(question) } + Button("Move up") { model.move(question, by: -1) } + Button("Move down") { model.move(question, by: 1) } + Divider() + Button("Delete", role: .destructive) { model.remove(question) } + } label: { + Image(systemName: "ellipsis.circle") + } + .menuStyle(.borderlessButton) + .fixedSize() + } + + TextField("Question the model reads", text: $question.text, axis: .vertical) + .lineLimit(1 ... 4) + + if question.kind != .noul { + optionsEditor + } + if question.kind == .score { + Label("Small models score worst on scales. Try yes/no or multiple choice.", systemImage: "info.circle") + .font(.jCaption) + .foregroundStyle(.secondary) + } + ForEach(question.problems, id: \.self) { p in + Label(p, systemImage: "exclamationmark.triangle") + .font(.jCaption) + .foregroundStyle(.orange) + } + } + .padding(4) + } + } + + private var optionsEditor: some View { + VStack(alignment: .leading, spacing: 4) { + ForEach(Array($question.options.enumerated()), id: \.element.id) { i, $opt in + HStack(spacing: 6) { + Text(question.kind == .choice ? String(UnicodeScalar(UInt8(65 + min(i, 25)))) : String(i)) + .font(.system(size: 12, design: .monospaced).weight(.bold)) + .frame(width: 18, height: 18) + .background(.tint.opacity(0.15), in: RoundedRectangle(cornerRadius: 4)) + if question.kind == .choice { + TextField("key", text: $opt.key) + .font(.system(size: 14, design: .monospaced)) + .frame(width: 110) + } + TextField("description the model reads", text: $opt.description) + Button { + question.options.removeAll { $0.id == opt.id } + } label: { + Image(systemName: "minus.circle") + } + .buttonStyle(.borderless) + .disabled(question.options.count <= 2) + } + } + Button { + let n = question.options.count + question.options.append(AnswerOption(key: question.kind == .score ? String(n) : "option\(n + 1)", description: "")) + } label: { + Label("Add option", systemImage: "plus.circle") + } + .buttonStyle(.borderless) + .font(.jCallout) + } + } + + private func fillOptionsIfNeeded(_ kind: QuestionKind) { + guard kind != .noul, question.options.count < 2 else { return } + question.options = kind == .score + ? [AnswerOption(key: "0", description: "low"), AnswerOption(key: "1", description: "medium"), AnswerOption(key: "2", description: "high")] + : [AnswerOption(key: "first", description: ""), AnswerOption(key: "second", description: "")] + } +} diff --git a/examples/jevdk/Sources/JevDK/Fonts.swift b/examples/jevdk/Sources/JevDK/Fonts.swift new file mode 100644 index 0000000..07e6af3 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/Fonts.swift @@ -0,0 +1,12 @@ +import SwiftUI + +/// JevDK's text sizes: macOS's standard sizes plus 2 pt, for readability at a desk. +/// (macOS doesn't scale its text styles from one setting, so every font in the app uses these.) +extension Font { + static let jBody = Font.system(size: 15) + static let jCallout = Font.system(size: 14) + static let jCaption = Font.system(size: 12) + static let jCaption2 = Font.system(size: 12) + static let jHeadline = Font.system(size: 15, weight: .semibold) + static let jTitle3 = Font.system(size: 17) +} diff --git a/examples/jevdk/Sources/JevDK/HostedBackends.swift b/examples/jevdk/Sources/JevDK/HostedBackends.swift new file mode 100644 index 0000000..40e5830 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/HostedBackends.swift @@ -0,0 +1,99 @@ +import Foundation +import LocalLMLabSDKCore +import LocalLMLabSDKRemote + +/// The hosted Jev backends JevDK offers, each an SDK `JevDecisionProvider` registered in the +/// lab under its own decision scheme. +enum HostedBackend: String, CaseIterable, Identifiable, Sendable { + case featherlessDemo, featherless, openRouter + + var id: String { rawValue } + + var displayName: String { + switch self { + case .featherlessDemo: "Featherless demo" + case .featherless: "Featherless" + case .openRouter: "OpenRouter · TypeSafe" + } + } + + var blurb: String { + switch self { + case .featherlessDemo: "Free, no key · 2k-token context, a few requests per second · token-scored like Local" + case .featherless: "featherless.ai key · token-scored like Local" + case .openRouter: "OpenRouter key · TypeSafe's decision-trained Jev" + } + } + + var needsKey: Bool { self != .featherlessDemo } + + /// Environment / .env variable for headless runs. + var keyVariable: String? { + switch self { + case .featherlessDemo: nil + case .featherless: "FEATHERLESS_API_KEY" + case .openRouter: "OPENROUTER_API_KEY" + } + } + + var defaultModel: String { + switch self { + case .featherlessDemo, .featherless: "featherless-ai/Qwen3.8-27B-classifier" + case .openRouter: "typesafe/jev-1.13" + } + } + + /// The SDK config, with a scheme of its own so the demo and the keyed Featherless endpoint + /// can both be registered in one lab. + func config(key: String?) -> JevProviderConfig { + var c: JevProviderConfig + switch self { + case .featherlessDemo: c = .featherlessDemo() + case .featherless: c = .featherless(key: key ?? "") + case .openRouter: c = .openRouter(key: key ?? "") + } + c.scheme = rawValue.lowercased() + return c + } + + var suggestedModels: [String] { config(key: nil).models } + + /// The free demo is rate-limited; space batch inputs out when it's on. + var minInterval: Duration { self == .featherlessDemo ? .milliseconds(600) : .zero } + + /// The demo publishes a model list; the others don't have a Jev-specific one. + var modelsURL: URL? { + self == .featherlessDemo ? URL(string: "https://simple-jev-demo-api.featherless.ai/v1/models") : nil + } + + func listModels() async -> [String] { + guard let url = modelsURL, + let (data, _) = try? await URLSession.shared.data(from: url), + let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any], + let list = obj["data"] as? [[String: Any]] + else { return suggestedModels } + return list.compactMap { $0["id"] as? String } + } + + func modelID(_ model: String) -> ModelID? { ModelID(scheme: rawValue.lowercased(), rest: model) } + + func label(model: String) -> String { + "\(displayName) · \(model.split(separator: "/").last.map(String.init) ?? model)" + } +} + +/// Reads an API key from the environment or a `.env` file, never printing it. Accepts +/// `NAME=value` lines or a file holding just the key. +enum KeyFile { + static func key(named name: String, file: URL = URL(fileURLWithPath: ".env")) -> String? { + if let v = ProcessInfo.processInfo.environment[name], !v.isEmpty { return v } + guard let text = try? String(contentsOf: file, encoding: .utf8) else { return nil } + let lines = text.split(whereSeparator: \.isNewline).map { $0.trimmingCharacters(in: .whitespaces) } + for line in lines where line.hasPrefix(name + "=") || line.hasPrefix("export " + name + "=") { + let v = line.split(separator: "=", maxSplits: 1).last.map(String.init) ?? "" + return v.trimmingCharacters(in: CharacterSet(charactersIn: "\"' ")) + } + let bare = lines.filter { !$0.isEmpty && !$0.hasPrefix("#") } + return bare.count == 1 && !bare[0].contains("=") ? bare[0] : nil + } +} diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift new file mode 100644 index 0000000..09655c6 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -0,0 +1,350 @@ +import AppKit +import LocalLMLabSDKCore +import LocalLMLabSDKInference +import LocalLMLabSDKRemote +import OpenJevKit +import SwiftUI + +@main +struct JevDKApp: App { + @State private var model = AppModel() + + init() { + // Headless checks, both through `lab.decide` like the window: + // `JevDK --check ` runs a preset's batch on a local model the SDK + // has downloaded and prints the answers, then exits. + let args = CommandLine.arguments + if let i = args.firstIndex(of: "--check"), args.count > i + 2 { + HeadlessCheck.run(preset: args[i + 1], modelDir: args[i + 2]) + } + // `JevDK --check-remote [model]`: the + // same, against a hosted Jev. Keys come from the environment or ./.env + // (FEATHERLESS_API_KEY / OPENROUTER_API_KEY) and are never printed. + if let i = args.firstIndex(of: "--check-remote"), args.count > i + 2 { + HeadlessCheck.runRemote(preset: args[i + 1], provider: args[i + 2], + model: args.count > i + 3 && !args[i + 3].hasPrefix("--") ? args[i + 3] : nil) + } + + // Launched with `swift run` there is no app bundle, so macOS starts this as a + // background-style process that never becomes active. Claim regular-app status + // (only if needed; re-setting it logs "Task policy set failed"). + if NSApplication.shared.activationPolicy() != .regular { + NSApplication.shared.setActivationPolicy(.regular) + } + DispatchQueue.main.async { NSApplication.shared.activate() } + } + + var body: some Scene { + WindowGroup("JevDK") { + ContentView(model: model) + .font(.jBody) + .frame(minWidth: 1060, minHeight: 660) + } + .commands { + CommandGroup(replacing: .newItem) { + Button("Open Question Set…") { model.open() }.keyboardShortcut("o") + } + CommandGroup(replacing: .saveItem) { + Button("Save") { model.save() }.keyboardShortcut("s") + Button("Save As…") { model.save(as: true) }.keyboardShortcut("s", modifiers: [.command, .shift]) + Divider() + Button("Export for App…") { model.exportForApp() }.keyboardShortcut("e") + Divider() + Button("Import Answer Set…") { model.importAnswerSet() } + Button("Export Answer Set…") { model.exportAnswerSet() } + } + } + } +} + +struct ContentView: View { + @Bindable var model: AppModel + @State private var showDownload = false + @State private var showBackends = false + /// An example waiting for "replace your work?" confirmation. + @State private var pendingPreset: Presets.Preset? + + /// The questions panel on the left; hiding it gives the batch grid the whole window. + @AppStorage("showQuestionsPanel") private var showEditor = true + + var body: some View { + HSplitView { + if showEditor { + EditorView(model: model) + .frame(minWidth: 380, idealWidth: 460) + } + ResultsView(model: model) + .frame(minWidth: 480) + } + .toolbar { + ToolbarItemGroup(placement: .navigation) { + Button { + showEditor.toggle() + } label: { + Label(showEditor ? "Hide questions" : "Show questions", systemImage: "sidebar.left") + } + .help(showEditor ? "Hide the questions panel to give results the whole window" : "Show the questions panel") + Menu { + ForEach(Presets.all) { p in + Button(p.set.name) { + if model.workspaceIsPristine { model.loadPreset(p) } else { pendingPreset = p } + } + } + } label: { + Label("Examples", systemImage: "square.stack") + } + } + ToolbarItemGroup(placement: .primaryAction) { + Picker("Model", selection: $model.selectedModelID) { + if model.models.isEmpty { Text("No models found").tag(String?.none) } + ForEach(model.models) { m in + Text("\(m.displayName) · \(m.sizeText)\(m.isMoE ? " · MoE ⚠︎" : "")") + .tag(Optional(m.id)) + } + } + .frame(width: 300) + .help(modelHelp) + Button { + model.refreshModels() + } label: { + Label("Rescan models", systemImage: "arrow.clockwise") + } + Button { + showDownload = true + } label: { + Label("Models", systemImage: "square.and.arrow.down.on.square") + } + .help("Choose, download or remove local models") + Button { + showBackends = true + } label: { + Label("Backends", systemImage: "square.3.layers.3d") + } + .help("Compare the local model with hosted Jev (Featherless, OpenRouter · TypeSafe)") + } + } + .safeAreaInset(edge: .bottom) { statusBar } + .sheet(isPresented: $showDownload) { ModelsSheet(model: model) } + .confirmationDialog("Replace your current questions with “\(pendingPreset?.set.name ?? "")”?", + isPresented: Binding(get: { pendingPreset != nil }, set: { if !$0 { pendingPreset = nil } })) { + Button("Save to a file first…") { + if let p = pendingPreset, model.save(as: true) { model.loadPreset(p) } + pendingPreset = nil + } + Button("Replace", role: .destructive) { + if let p = pendingPreset { model.loadPreset(p) } + pendingPreset = nil + } + Button("Cancel", role: .cancel) { pendingPreset = nil } + } message: { + Text("Your questions, system instructions, inputs, marked answers and calibration are replaced. JevDK only keeps your latest workspace, so save it to a file (⌘S) if you want it back.") + } + .onReceive(NotificationCenter.default.publisher(for: NSApplication.willTerminateNotification)) { _ in + model.persistNow() + } + .sheet(isPresented: $showBackends) { BackendsSheet(model: model) } + } + + private var modelHelp: String { + guard let m = model.selectedModel else { return "No SDK-verified models yet: open Models to choose one." } + var s = "\(m.repoID) · \(m.sizeText) · verified by the SDK" + if m.isMoE { + s += "\nMixture-of-experts: not recommended as a decider. Its probabilities can shift with how the prompt is split." + } + return s + } + + private var statusBar: some View { + HStack(spacing: 6) { + if model.useLocal, let m = model.selectedModel { + Text(m.repoID) + if m.isMoE { + Text("· MoE: not recommended as a decider").foregroundStyle(.orange) + } + } + Spacer() + let hosted = model.enabledHosted + if !hosted.isEmpty { + Label("Inputs are sent to \(hosted.map(\.displayName).joined(separator: ", "))", systemImage: "network") + .foregroundStyle(.secondary) + } else { + Text("Local only · nothing leaves this Mac").foregroundStyle(.secondary) + } + Text("· via lab.decide").foregroundStyle(.tertiary) + } + .font(.jCaption) + .padding(.horizontal, 12) + .padding(.vertical, 5) + .background(.bar) + } +} + +/// `--check` / `--check-remote`: a preset's batch through `lab.decide`, printed. +@MainActor +enum HeadlessCheck { + static func runRemote(preset name: String, provider: String, model: String?) -> Never { + guard let p = preset(name), let backend = HostedBackend(rawValue: provider) else { + print("unknown preset or provider (have: \(HostedBackend.allCases.map(\.rawValue)))"); exit(1) + } + let key = backend.keyVariable.flatMap { KeyFile.key(named: $0) } + if backend.needsKey { print("key: \(key == nil ? "MISSING" : "found (\(key!.count) chars)")") } + let lab = LocalLMLab() + try? lab.models.register(decision: JevDecisionProvider(backend.config(key: key))) + let modelName = model ?? backend.defaultModel + lab.models.route(decision: "check", to: backend.modelID(modelName)!) + print(backend.label(model: modelName)) + run(lab: lab, preset: p, pause: backend.minInterval, + info: .init(backend: backend.rawValue, model: modelName, revision: nil, sizeBytes: nil, isMoE: nil, + wrapperLabel: nil, wrapperSystem: nil)) + } + + static func run(preset name: String, modelDir repo: String) -> Never { + guard let p = preset(name) else { print("unknown preset \(name)"); exit(1) } + let mlx = MLXModelProvider() + let openjev = OpenJevDecisionProvider(mlx: mlx, wrapper: OpenJevWrapper(system: p.set.system, inputLabel: p.set.inputLabel)) + let id = AppModel.openJevID(repo) + guard openjev.availability(for: id).isAvailable else { + print("\(repo) isn't available to the SDK (\(openjev.availability(for: id))); download or verify it first."); exit(1) + } + print("\(repo)\(openjev.warnings(for: id).isEmpty ? "" : " · \(openjev.warnings(for: id))")") + let lab = LocalLMLab() + try? lab.models.register(decision: openjev) + lab.models.route(decision: "check", to: id) + let installed = mlx.installed.first { $0.repoID == repo } + run(lab: lab, preset: p, pause: .zero, + info: .init(backend: "local", model: repo, revision: installed?.resolvedRevision, sizeBytes: installed?.sizeBytes, + isMoE: openjev.warnings(for: id).contains(.mixtureOfExperts), + wrapperLabel: p.set.inputLabel, wrapperSystem: p.set.system)) + } + + private static func preset(_ name: String) -> Presets.Preset? { + Presets.all.first { $0.set.name.lowercased() == name.lowercased() } + } + + /// `--csv ` after either check appends the run to a results CSV, as the window's + /// Results CSV button does (no marked answers headless, so `expected` / `correct` are empty). + private static func run(lab: LocalLMLab, preset p: Presets.Preset, pause: Duration, info: ResultsCSV.BackendInfo, + setup: @escaping @Sendable () async -> Void = {}) -> Never { + let args = CommandLine.arguments + let csv = args.firstIndex(of: "--csv").flatMap { args.count > $0 + 1 ? URL(fileURLWithPath: args[$0 + 1]) : nil } + let started = Date() + let runInfo = ResultsCSV.RunInfo(runID: ResultsCSV.newRunID(at: started), savedAt: started, + sdkVersion: LocalLMLabSDKVersion.current, machine: ResultsCSV.machine, + questionSet: p.set.name) + Task { @MainActor in + await setup() + var lines: [String] = [] + let questions = p.set.questions.map(\.sdkQuestion) + for line in p.batch.split(separator: "\n").map(String.init) where !line.isEmpty { + do { + let d = try await lab.decide(route: "check", state: .text(line), questions: questions) + let rows = QuestionResult.rows(for: p.set.questions, decision: d) + let result = DecisionResult(input: line, results: rows, sharedPrefixTokens: 0, + milliseconds: Double(d.latency.components.seconds) * 1000 + Double(d.latency.components.attoseconds) / 1e15, + backend: info.model) + lines += ResultsCSV.rows(run: runInfo, backend: info, result: result, labels: [:], + inputTokens: d.usage?.inputTokens, costUSD: d.usage?.cost) + let cells = rows.map { "\($0.question.name)=\($0.answer) \(Int(($0.confidence * 100).rounded()))%" } + let ms = Double(d.latency.components.seconds) * 1000 + Double(d.latency.components.attoseconds) / 1e15 + print(String(format: "%5.0f ms ", ms) + line.prefix(60) + "\n " + cells.joined(separator: " ") + + (d.usage?.summary.map { "\n [\($0)]" } ?? "")) + } catch { + print("error: \(error.localizedDescription)") + break + } + if pause > .zero { try? await Task.sleep(for: pause) } + } + if let csv { + do { + try ResultsCSV.append(lines, to: csv) + print("appended \(lines.count) rows to \(csv.path) (run \(runInfo.runID))") + } catch { + print("csv: \(error.localizedDescription)") + } + } + exit(0) + } + RunLoop.main.run() + exit(0) + } +} + +struct BackendsSheet: View { + @Bindable var model: AppModel + @Environment(\.dismiss) private var dismiss + @State private var keyDrafts: [HostedBackend: String] = [:] + + var body: some View { + VStack(alignment: .leading, spacing: 14) { + Text("Backends").font(.jHeadline) + Text("Every backend that's on answers the same questions through lab.decide, shown side by side. Hosted backends receive the input text and the questions.") + .font(.jCallout).foregroundStyle(.secondary) + + GroupBox { + Toggle(isOn: $model.useLocal) { + VStack(alignment: .leading) { + Text("Local").fontWeight(.medium) + Text("openjev:\(model.selectedModel?.repoID ?? "no model") · token-scored · on this Mac").font(.jCaption).foregroundStyle(.secondary) + } + } + .frame(maxWidth: .infinity, alignment: .leading) + .padding(4) + } + + ForEach(HostedBackend.allCases) { b in + GroupBox { + VStack(alignment: .leading, spacing: 8) { + Toggle(isOn: Binding(get: { model.hostedEnabled[b] ?? false }, set: { model.hostedEnabled[b] = $0 })) { + VStack(alignment: .leading) { + Text(b.displayName).fontWeight(.medium) + Text(b.blurb).font(.jCaption).foregroundStyle(.secondary) + } + } + HStack { + Text("Model").font(.jCallout) + TextField("model id", text: Binding(get: { model.hostedModel[b] ?? b.defaultModel }, + set: { model.hostedModel[b] = $0 })) + .textFieldStyle(.roundedBorder) + Menu { + ForEach(model.hostedModelChoices[b] ?? b.suggestedModels, id: \.self) { m in + Button(m) { model.hostedModel[b] = m } + } + if b.modelsURL != nil { + Divider() + Button("Refresh list") { Task { await model.refreshHostedModels(b) } } + } + } label: { Image(systemName: "list.bullet") } + .menuStyle(.borderlessButton) + .fixedSize() + } + if b.needsKey { + HStack { + Text("API key").font(.jCallout) + SecureField(model.apiKey(b) == nil ? "paste key" : "saved in Keychain", + text: Binding(get: { keyDrafts[b] ?? "" }, set: { keyDrafts[b] = $0 })) + .textFieldStyle(.roundedBorder) + Button("Save") { + model.setAPIKey(keyDrafts[b]?.trimmingCharacters(in: .whitespacesAndNewlines), b) + keyDrafts[b] = "" + } + .disabled((keyDrafts[b] ?? "").isEmpty) + if model.apiKey(b) != nil { + Button("Forget") { model.setAPIKey(nil, b); keyDrafts[b] = "" } + } + } + } + } + .padding(4) + } + } + + HStack { + Spacer() + Button("Done") { dismiss() }.keyboardShortcut(.defaultAction) + } + } + .padding(20) + .frame(width: 620) + .font(.jBody) + } +} diff --git a/examples/jevdk/Sources/JevDK/Keychain.swift b/examples/jevdk/Sources/JevDK/Keychain.swift new file mode 100644 index 0000000..d347e51 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/Keychain.swift @@ -0,0 +1,42 @@ +import Foundation +import Security + +/// API keys for remote Jev providers, in the login Keychain. Never written to disk or logs. +enum Keychain { + private static let service = "JevDK" + /// The app's earlier names (AskJev, then JevLab); keys saved under them move across on first read. + private static let formerServices = ["JevLab", "AskJev"] + + static func get(_ account: String) -> String? { + if let value = read(account, service: service) { return value } + for former in formerServices { + guard let old = read(account, service: former) else { continue } + set(old, for: account) + delete(account, service: former) + return old + } + return nil + } + + static func set(_ value: String?, for account: String) { + delete(account, service: service) + guard let value, !value.isEmpty else { return } + let add: [String: Any] = [kSecClass as String: kSecClassGenericPassword, kSecAttrService as String: service, + kSecAttrAccount as String: account, kSecValueData as String: Data(value.utf8)] + SecItemAdd(add as CFDictionary, nil) + } + + private static func read(_ account: String, service: String) -> String? { + let q: [String: Any] = [kSecClass as String: kSecClassGenericPassword, kSecAttrService as String: service, + kSecAttrAccount as String: account, kSecReturnData as String: true] + var out: AnyObject? + guard SecItemCopyMatching(q as CFDictionary, &out) == errSecSuccess, let data = out as? Data else { return nil } + return String(data: data, encoding: .utf8) + } + + private static func delete(_ account: String, service: String) { + let q: [String: Any] = [kSecClass as String: kSecClassGenericPassword, kSecAttrService as String: service, + kSecAttrAccount as String: account] + SecItemDelete(q as CFDictionary) + } +} diff --git a/examples/jevdk/Sources/JevDK/ModelsSheet.swift b/examples/jevdk/Sources/JevDK/ModelsSheet.swift new file mode 100644 index 0000000..bb9bd90 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/ModelsSheet.swift @@ -0,0 +1,205 @@ +import LocalLMLabSDKCore +import OpenJevKit +import SwiftUI + +/// Choose a local model: what's installed, recommended deciders, a Hugging Face search, and +/// copies in the cache to verify. Everything goes through the SDK's `MLXModelProvider`: it checks a +/// repo before downloading (reachable, MLX format, supported architecture, size vs memory and +/// disk), then downloads and verifies. +struct ModelsSheet: View { + @Bindable var model: AppModel + @Environment(\.dismiss) private var dismiss + @State private var tab = Tab.installed + @State private var query = "" + @State private var hits: [AppModel.HubModel] = [] + @State private var searching = false + @State private var confirmRemove: String? + + enum Tab: String, CaseIterable { case installed = "Installed", recommended = "Recommended", search = "Search Hugging Face" } + + var body: some View { + VStack(alignment: .leading, spacing: 12) { + Text("Models").font(.jHeadline) + Picker("", selection: $tab) { + ForEach(Tab.allCases, id: \.self) { Text($0.rawValue).tag($0) } + } + .pickerStyle(.segmented) + .labelsHidden() + + Group { + switch tab { + case .installed: installed + case .recommended: recommended + case .search: search + } + } + .frame(minHeight: 260, alignment: .top) + + Divider() + downloadPanel + + HStack { + if let e = model.errorMessage { Text(e).font(.jCaption).foregroundStyle(.orange).lineLimit(2) } + Spacer() + Button("Done") { dismiss() }.keyboardShortcut(.defaultAction) + } + } + .padding(20) + .frame(width: 680) + .font(.jBody) + .onAppear { model.refreshModels() } + .confirmationDialog("Remove \(confirmRemove ?? "")?", isPresented: Binding(get: { confirmRemove != nil }, set: { if !$0 { confirmRemove = nil } })) { + Button("Remove", role: .destructive) { if let r = confirmRemove { model.remove(r) }; confirmRemove = nil } + } message: { + Text("Deletes its weights from disk. You can download it again later.") + } + } + + // MARK: Tabs + + private var installed: some View { + ScrollView { + VStack(alignment: .leading, spacing: 6) { + if model.models.isEmpty { + Text("No models downloaded through the SDK yet. Pick one under Recommended, or search.") + .font(.jCallout).foregroundStyle(.secondary) + } + ForEach(model.models) { m in + HStack { + VStack(alignment: .leading) { + Text(m.repoID).font(.system(size: 14, design: .monospaced)) + Text("\(m.sizeText)\(m.isMoE ? " · mixture-of-experts: not recommended as a decider" : "")") + .font(.jCaption).foregroundStyle(m.isMoE ? .orange : .secondary) + } + Spacer() + if model.selectedModelID == m.id { + Label("In use", systemImage: "checkmark").font(.jCaption) + } else { + Button("Use") { model.selectedModelID = m.id } + } + Button(role: .destructive) { confirmRemove = m.repoID } label: { Image(systemName: "trash") } + .buttonStyle(.borderless) + .help("Remove from disk") + } + } + if !model.unverifiedRepos.isEmpty { + Divider().padding(.vertical, 4) + Text("In the Hugging Face cache, not yet verified by the SDK").font(.jCallout.weight(.medium)) + Text("Fetched by another tool. Verifying checks the files and downloads only what's missing.") + .font(.jCaption).foregroundStyle(.secondary) + ForEach(model.unverifiedRepos, id: \.self) { repo in + HStack { + Text(repo).font(.system(size: 14, design: .monospaced)) + Spacer() + Button("Verify") { Task { await model.download(repo) } } + .disabled(model.downloadProgress != nil) + } + } + } + } + } + } + + private var recommended: some View { + VStack(alignment: .leading, spacing: 6) { + Text("Small dense instruct models suit a decider. Mixture-of-experts models gave unstable probabilities in testing.") + .font(.jCaption).foregroundStyle(.secondary) + ForEach(AppModel.recommended, id: \.repo) { r in + HStack { + VStack(alignment: .leading) { + Text(r.repo).font(.system(size: 14, design: .monospaced)) + Text(r.note).font(.jCaption).foregroundStyle(.secondary) + } + Spacer() + if model.models.contains(where: { $0.repoID == r.repo }) { + Label("Installed", systemImage: "checkmark").font(.jCaption) + } else { + Button("Select") { select(r.repo) } + } + } + } + } + } + + private var search: some View { + VStack(alignment: .leading, spacing: 6) { + HStack { + TextField("Search MLX models, e.g. “qwen3 4b” or “llama 3b instruct”", text: $query) + .textFieldStyle(.roundedBorder) + .onSubmit { runSearch() } + Button("Search") { runSearch() }.disabled(query.trimmingCharacters(in: .whitespaces).isEmpty) + if searching { ProgressView().controlSize(.small) } + } + ScrollView { + VStack(alignment: .leading, spacing: 4) { + ForEach(hits) { h in + HStack { + VStack(alignment: .leading) { + Text(h.id).font(.system(size: 14, design: .monospaced)) + Text("\(h.pipeline_tag ?? "?") · \(h.downloads ?? 0) downloads") + .font(.jCaption).foregroundStyle(.secondary) + } + Spacer() + Button("Select") { select(h.id) } + } + } + } + } + Text("Text-generation models fit best; image-text models load too but run slower.") + .font(.jCaption).foregroundStyle(.secondary) + } + } + + // MARK: Check → download + + private var downloadPanel: some View { + VStack(alignment: .leading, spacing: 8) { + HStack { + TextField("org/model", text: $model.downloadRepo) + .textFieldStyle(.roundedBorder) + .onSubmit { Task { await model.checkRepo() } } + .onChange(of: model.downloadRepo) { _, _ in model.preflight = nil } + Button("Check") { Task { await model.checkRepo() } } + .disabled(!model.downloadRepo.contains("/")) + Button("Download") { + Task { await model.download(); tab = .installed } + } + .buttonStyle(.borderedProminent) + .disabled(model.preflight?.passed != true || model.downloadProgress != nil) + } + HStack { + if let p = model.preflight { + if let w = p.weightBytes { + Text("Size \(ByteCountFormatter.string(fromByteCount: w, countStyle: .file))") + } + if p.passed { + Label("SDK checks passed", systemImage: "checkmark.circle").foregroundStyle(.green) + } else { + Label(p.detail ?? "A check failed", systemImage: "exclamationmark.triangle").foregroundStyle(.orange) + } + } else { + Text("Select a model above or type a repo id, then Check: the SDK confirms it's an MLX model this Mac can run before anything downloads.") + .foregroundStyle(.secondary) + } + Spacer() + if let f = model.downloadProgress { ProgressView(value: f).frame(width: 140) } + } + .font(.jCaption) + } + } + + private func select(_ repo: String) { + model.downloadRepo = repo + Task { await model.checkRepo() } + } + + private func runSearch() { + let q = query.trimmingCharacters(in: .whitespaces) + guard !q.isEmpty else { return } + searching = true + Task { + hits = await model.searchHub(q) + searching = false + } + } +} diff --git a/examples/jevdk/Sources/JevDK/Presets.swift b/examples/jevdk/Sources/JevDK/Presets.swift new file mode 100644 index 0000000..046f0b2 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/Presets.swift @@ -0,0 +1,226 @@ +import OpenJevKit + +/// Starting points, taken from the question sets the SDK's OpenJev evaluation measured. The wordings +/// are the best-scoring variants on Qwen3-4B-4bit as of 2026-10-03. +enum Presets { + struct Preset: Identifiable { + var id: String { self.set.name } + let set: QuestionSet + let input: String + let batch: String + /// One-click sample inputs for single-input mode: (chip label, text). + var samples: [(String, String)] = [] + } + + /// A preset built from named samples: the first is the input, all of them are the batch. + static func scenario(_ set: QuestionSet, _ samples: [(String, String)]) -> Preset { + Preset(set: set, input: samples[0].1, batch: samples.map(\.1).joined(separator: "\n"), samples: samples) + } + + static let all: [Preset] = [ + Preset(set: triage, input: triageInput, batch: triageBatch), + Preset(set: tools, input: "Will I need an umbrella in Boston this afternoon?", batch: toolsBatch), + Preset(set: answerCheck, input: "Question: What is the capital of Australia?\nAnswer: Sydney.", batch: answerCheckBatch), + scenario(support, supportSamples), + scenario(reviews, reviewSamples), + scenario(moderation, moderationSamples), + Preset(set: QuestionSet(name: "Blank", system: QuestionSet.defaultSystem, questions: []), input: "", batch: ""), + ] + + // MARK: Router triage + + static let triage = QuestionSet( + name: "Router triage", + // Exactly the evaluation's triage system prompt, so results match it. + system: """ + You are a classifier. Read the user's request and answer one multiple-choice question about it. \ + Reply with the answer label only, nothing else. + """, + inputLabel: "Request", + questions: [ + EditableQuestion(name: "needsLiveData", kind: .noul, text: "Does this user query need live data?"), + EditableQuestion(name: "kind", kind: .choice, text: "Which category fits this request best?", options: [ + AnswerOption(key: "factual", description: "a question about facts, how something works, or advice"), + AnswerOption(key: "code", description: "writing, fixing or explaining code or software"), + AnswerOption(key: "math", description: "a calculation, equation or proof"), + AnswerOption(key: "creative", description: "a writing or language task (stories, poems, emails, rewriting, summarising, translating)"), + AnswerOption(key: "chitchat", description: "small talk or social conversation, no real task"), + ]), + EditableQuestion(name: "sensitive", kind: .noul, + text: "Does the request contain personal or private information about the user or another person?"), + EditableQuestion(name: "depth", kind: .score, text: "How much reasoning does answering this need?", options: [ + AnswerOption(key: "0", description: "none (recall or rewrite)"), + AnswerOption(key: "1", description: "a little"), + AnswerOption(key: "2", description: "multi-step"), + AnswerOption(key: "3", description: "deep (hard problem)"), + ]), + ]) + + static let triageInput = "Who won last night's NBA game between the Lakers and the Celtics?" + + static let triageBatch = """ + What is the capital of Australia? + Who won last night's NBA game between the Lakers and the Celtics? + What's the current price of Bitcoin? + What's the population of Tokyo? + Write a Swift function that merges two sorted arrays in O(n). + Prove that the square root of 2 is irrational. + Write a haiku about the first snow. + Summarize this in one sentence: "The committee agreed to raise park funding by 10%." + My blood test says my ALT is 85 U/L. Is that bad? + What is the normal range for ALT in a blood test? + hey, how's it going? + When did the Berlin Wall fall? + """ + + // MARK: Tool choice + + static let tools = QuestionSet( + name: "Tool choice", + system: """ + You are a classifier. Read the user's request and answer one multiple-choice question about it. \ + Reply with the answer label only, nothing else. + + The assistant has these tools: + - calendar: read, create or change events in the user's calendar + - weather: forecasts and current conditions for a place + - web_search: search the web for current or niche information + - files: search the user's own documents and files + - calculator: exact arithmetic + - none: answer directly without a tool + """, + inputLabel: "Request", + questions: [ + EditableQuestion(name: "tool", kind: .choice, text: "Which tool should the assistant use first?", options: [ + AnswerOption(key: "calendar", description: "calendar"), + AnswerOption(key: "weather", description: "weather"), + AnswerOption(key: "web_search", description: "web_search"), + AnswerOption(key: "files", description: "files"), + AnswerOption(key: "calculator", description: "calculator"), + AnswerOption(key: "none", description: "none"), + ]), + ]) + + static let toolsBatch = """ + Schedule a dentist appointment for next Tuesday at 3pm. + Will I need an umbrella in Boston this afternoon? + Who won the Oscar for best picture this year? + Find the PDF of my lease agreement. + What is the capital of Peru? + Compute the monthly payment on a $300,000 mortgage at 6.5% over 30 years. + Summarize the causes of the French Revolution. + """ + + // MARK: Answer check + + static let answerCheck = QuestionSet( + name: "Answer check", + system: """ + You are a strict grader. Read the question and the proposed answer in the input, then \ + answer one yes/no question about them. Reply with Yes or No only. + """, + questions: [ + EditableQuestion(name: "complete", kind: .noul, + text: "Does the proposed answer fully and correctly address the question?"), + ]) + + static let answerCheckBatch = """ + Question: What is the capital of Australia? Answer: Canberra. + Question: What is the capital of Australia? Answer: Sydney. + Question: What's 17 × 23? Answer: 391. + Question: What's 17 × 23? Answer: 381. + Question: What's the weather in Seattle right now? Answer: I don't have access to live weather data. + Question: List three causes of World War I. Answer: The assassination of Archduke Franz Ferdinand. + """ + + // MARK: Everyday triage (after Featherless's Simple Jev playground scenarios) + // + // Questions ask about what the text says ("explicitly asks for…", "mentions…"), so they can be + // judged from the input alone. + + static let support = QuestionSet( + name: "Customer support", + system: """ + You are a classifier. Read the customer's message and answer one multiple-choice question \ + about it. Reply with the answer label only, nothing else. + """, + inputLabel: "Customer message", + questions: [ + EditableQuestion(name: "team", kind: .choice, text: "Which team should handle this customer message?", options: [ + AnswerOption(key: "billing", description: "payments, invoices and refunds"), + AnswerOption(key: "technical", description: "bugs, outages and product errors"), + AnswerOption(key: "account", description: "profile, login and subscription changes"), + ]), + EditableQuestion(name: "urgent", kind: .noul, + text: "Is the customer blocked from using the product right now?"), + EditableQuestion(name: "refund", kind: .noul, text: "Does the customer explicitly ask for money back?"), + EditableQuestion(name: "angry", kind: .noul, text: "Is the customer angry or threatening to leave?"), + ]) + + static let supportSamples: [(String, String)] = [ + ("Double charge", "I see two charges for my Pro plan on this month's statement. Please refund one of them. Otherwise the app is great."), + ("App outage", "Nothing loads since this morning, just a spinning wheel on every page. My whole team is stuck. What's going on?"), + ("Account update", "How do I change the email address on my account? I'm moving to a new work address."), + ("Cancel threat", "Third time this month the sync lost my files. If it happens again I'm cancelling and switching to a competitor."), + ("Login trouble", "The password reset link says it has expired every time I click it, so I can't sign in."), + ] + + static let reviews = QuestionSet( + name: "Product reviews", + system: """ + You are a classifier. Read the product review and answer one multiple-choice question about \ + it. Reply with the answer label only, nothing else. + """, + inputLabel: "Review", + questions: [ + EditableQuestion(name: "sentiment", kind: .choice, text: "What is the overall sentiment of the review?", options: [ + AnswerOption(key: "positive", description: "mostly happy with the product"), + AnswerOption(key: "mixed", description: "both real praise and real complaints"), + AnswerOption(key: "negative", description: "mostly unhappy with the product"), + ]), + EditableQuestion(name: "topic", kind: .choice, text: "What does the review mainly talk about?", options: [ + AnswerOption(key: "quality", description: "how well the product works or how well it's made"), + AnswerOption(key: "price", description: "price or value for money"), + AnswerOption(key: "shipping", description: "delivery, packaging or arrival condition"), + AnswerOption(key: "service", description: "customer service or returns"), + ]), + EditableQuestion(name: "defect", kind: .noul, text: "Does the reviewer report a broken or faulty item?"), + EditableQuestion(name: "recommends", kind: .noul, text: "Does the reviewer say they would recommend it to others?"), + ]) + + static let reviewSamples: [(String, String)] = [ + ("Love it", "Best kettle I've owned. Boils fast, quiet, and looks great on the counter. Already bought one for my sister."), + ("Broken on arrival", "Box was crushed and the glass lid arrived cracked. Waiting on a replacement."), + ("Mixed", "Sound quality is excellent for the price, but the ear cushions started peeling after two months."), + ("Pricey", "Works fine, nothing special. Not worth twice the price of the store brand."), + ("Great support", "The blender died after a week, but support sent a new one within two days, no questions asked. Would recommend."), + ] + + static let moderation = QuestionSet( + name: "Community moderation", + system: """ + You are a content moderator for a friendly gardening forum. Read the post and answer one \ + multiple-choice question about it. Reply with the answer label only, nothing else. + """, + inputLabel: "Post", + questions: [ + EditableQuestion(name: "issue", kind: .choice, text: "This post was made on a gardening forum. Which problem, if any, does it have?", options: [ + AnswerOption(key: "none", description: "no problem, fine to publish"), + AnswerOption(key: "harassment", description: "insults or attacks another member"), + AnswerOption(key: "spam", description: "advertising, links to sell something, or repeated junk"), + AnswerOption(key: "personalInfo", description: "shares someone's private contact details or address"), + AnswerOption(key: "offTopic", description: "unrelated to gardening"), + ]), + EditableQuestion(name: "humanReview", kind: .noul, + text: "Is this post borderline enough that a human moderator should look at it?"), + ]) + + static let moderationSamples: [(String, String)] = [ + ("Helpful", "Tip for anyone struggling with seedlings: put a small fan on them for an hour a day. Stems got much sturdier."), + ("Insult", "Only an idiot would water succulents every day. Did you even read the pinned post?"), + ("Spam", "BEST garden tools 70% OFF!!! Click my profile link, today only!!!"), + ("Doxxing", "The guy who sold me those bad seeds is Mark, he lives at 18 Oak Lane and his number is 555-0199."), + ("Off topic", "Anyone watching the football tonight? Who do you think wins?"), + ] +} + diff --git a/examples/jevdk/Sources/JevDK/ResultsCSV.swift b/examples/jevdk/Sources/JevDK/ResultsCSV.swift new file mode 100644 index 0000000..33e2bf1 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/ResultsCSV.swift @@ -0,0 +1,149 @@ +import Foundation +import OpenJevKit + +// "Append results to CSV" for a batch run: one row per input × question × backend, appended to an +// existing results CSV so runs of different models, quantizations, wrappers or question wordings +// line up in one spreadsheet. Same conventions as the LocalLM Lab benchmark CSV: fixed header, +// append only to a file with exactly this header, never touch any other file, an unmeasured value +// is an empty cell. Average `correct` (yes = 1) by `model` and `question` for per-question accuracy. + +enum ResultsCSV { + static let columns = [ + "run_id", "saved_at", "app", "sdk_version", "machine", "question_set", + "backend", "model", "model_revision", "model_size_gb", "moe", "wrapper_label", "wrapper_system", "calibrated", + "input", "question", "question_kind", "question_text", + "expected", "answer", "correct", "confidence", "probabilities", "label_mass", + "question_ms", "decision_ms", "input_tokens", "cost_usd", + ] + + static var header: String { columns.joined(separator: ",") } + + /// A run id: the local date and time the batch started, plus a short suffix so two runs in the + /// same second stay apart, e.g. "2026-10-03 14:32:05 · a1b2". + static func newRunID(at date: Date) -> String { + let f = DateFormatter() + f.locale = Locale(identifier: "en_US_POSIX") + f.dateFormat = "yyyy-MM-dd HH:mm:ss" + return "\(f.string(from: date)) · \(String(UUID().uuidString.prefix(4)).lowercased())" + } + + /// How many characters of an input, question text or system prompt a row keeps. + static let previewLength = 200 + + /// What a run had in common, stamped on every row. + struct RunInfo { + let runID: String + let savedAt: Date + let sdkVersion: String + let machine: String + let questionSet: String + } + + /// What one backend was, for its rows. + struct BackendInfo { + let backend: String + let model: String + let revision: String? + let sizeBytes: Int64? + let isMoE: Bool? + let wrapperLabel: String? + let wrapperSystem: String? + } + + static func rows(run: RunInfo, backend: BackendInfo, result: DecisionResult, labels: [String: String], + inputTokens: Int?, costUSD: Double?) -> [String] { + let iso = ISO8601DateFormatter() + iso.formatOptions = [.withInternetDateTime] + iso.timeZone = .current + return result.results.map { r in + let expected = labels[r.question.name] + let calibrated = r.fidelity == .tokenScored(calibrated: true) + let cells: [String] = [ + run.runID, iso.string(from: run.savedAt), "JevDK", run.sdkVersion, run.machine, run.questionSet, + backend.backend, backend.model, backend.revision ?? "", + backend.sizeBytes.map { String(format: "%.2f", Double($0) / 1e9) } ?? "", + backend.isMoE.map { $0 ? "yes" : "no" } ?? "", + backend.wrapperLabel ?? "", backend.wrapperSystem.map(preview) ?? "", calibrated ? "yes" : "no", + preview(result.input), r.question.name, r.question.kind.rawValue, preview(r.question.text), + expected ?? "", r.answer, expected.map { $0 == r.answer ? "yes" : "no" } ?? "", + String(format: "%.4f", r.confidence), + zip(r.keys, r.probabilities).map { "\($0)=\(String(format: "%.4f", $1))" }.joined(separator: "; "), + backend.wrapperLabel == nil ? "" : String(format: "%.4f", r.labelMass), + String(format: "%.1f", r.milliseconds), String(format: "%.1f", result.milliseconds), + inputTokens.map(String.init) ?? "", costUSD.map { String(format: "%.6f", $0) } ?? "", + ] + return cells.map(escape).joined(separator: ",") + } + } + + static func preview(_ s: String) -> String { + String(s.split(whereSeparator: \.isNewline).joined(separator: " ").prefix(previewLength)) + } + + /// RFC 4180: quote a cell that holds a comma, quote or line break; double any quotes inside. + static func escape(_ cell: String) -> String { + guard cell.contains(where: { $0 == "," || $0 == "\"" || $0.isNewline }) else { return cell } + return "\"" + cell.replacingOccurrences(of: "\"", with: "\"\"") + "\"" + } + + enum WritePlan: Equatable { + /// No file, or an empty one: write the header, then the rows. + case newFile + /// A JevDK results CSV: append (after a line break, if the file lacks a final one). + case append(needsLineBreak: Bool) + /// Some other CSV, or one with different columns: don't touch it. + case mismatch + } + + static func plan(existing contents: String?) -> WritePlan { + guard let contents, !contents.isEmpty else { return .newFile } + // Up to any line break: "\r\n" (how Excel saves CSV) is one Character in Swift. + let firstLine = String(contents.prefix { !$0.isNewline }) + guard firstLine == header else { return .mismatch } + return .append(needsLineBreak: !(contents.last?.isNewline ?? false)) + } + + /// Append `lines` to `url` (creating it with the header), or throw if it's some other file. + static func append(_ lines: [String], to url: URL) throws { + let existing = try? String(contentsOf: url, encoding: .utf8) + var text: String + switch plan(existing: existing) { + case .newFile: + text = header + "\n" + case .append(let needsLineBreak): + text = needsLineBreak ? "\n" : "" + case .mismatch: + throw CSVError.mismatch(url.lastPathComponent) + } + text += lines.joined(separator: "\n") + "\n" + if existing == nil || existing?.isEmpty == true { + try text.write(to: url, atomically: true, encoding: .utf8) + } else { + let handle = try FileHandle(forWritingTo: url) + defer { try? handle.close() } + try handle.seekToEnd() + try handle.write(contentsOf: Data(text.utf8)) + } + } + + enum CSVError: LocalizedError { + case mismatch(String) + var errorDescription: String? { + switch self { + case .mismatch(let name): + "\(name) isn't a JevDK results file (its header is different), so it was left untouched. Choose another file or a new one." + } + } + } + + /// e.g. "Mac17,14 · 64 GB". + static var machine: String { + var size = 0 + sysctlbyname("hw.model", nil, &size, nil, 0) + var buf = [CChar](repeating: 0, count: max(size, 1)) + sysctlbyname("hw.model", &buf, &size, nil, 0) + let model = String(cString: buf) + let ram = Int((Double(ProcessInfo.processInfo.physicalMemory) / 1_073_741_824).rounded()) + return "\(model) · \(ram) GB" + } +} diff --git a/examples/jevdk/Sources/JevDK/ResultsView.swift b/examples/jevdk/Sources/JevDK/ResultsView.swift new file mode 100644 index 0000000..67c28a1 --- /dev/null +++ b/examples/jevdk/Sources/JevDK/ResultsView.swift @@ -0,0 +1,490 @@ +import OpenJevKit +import SwiftUI + +/// Right pane: one input with full detail, or many inputs as a grid. Every enabled backend +/// (local, Featherless, OpenRouter · TypeSafe) answers the same questions, shown together. +struct ResultsView: View { + @Bindable var model: AppModel + @State private var mode = Mode.single + @State private var showCalibration = false + + enum Mode: String, CaseIterable { case single = "Single input", batch = "Batch" } + + var body: some View { + VStack(spacing: 0) { + Picker("", selection: $mode) { + ForEach(Mode.allCases, id: \.self) { Text($0.rawValue).tag($0) } + } + .pickerStyle(.segmented) + .labelsHidden() + .padding([.horizontal, .top]) + + if let msg = model.errorMessage { + Label(msg, systemImage: "exclamationmark.triangle.fill") + .foregroundStyle(.orange) + .font(.jCallout) + .frame(maxWidth: .infinity, alignment: .leading) + .padding(.horizontal) + .padding(.top, 8) + } + + switch mode { + case .single: single + case .batch: batch + } + } + } + + // MARK: Single + + private var single: some View { + ScrollView { + VStack(alignment: .leading, spacing: 12) { + HStack(spacing: 6) { + Text("Input").font(.jCallout.weight(.medium)) + Spacer() + ForEach(model.samples.indices, id: \.self) { i in + Button(model.samples[i].0) { + model.input = model.samples[i].1 + Task { await model.run() } + } + .buttonStyle(.bordered) + .controlSize(.small) + .tint(model.input == model.samples[i].1 ? .accentColor : nil) + } + } + TextEditor(text: $model.input) + .font(.jBody) + .frame(minHeight: 70, maxHeight: 140) + .scrollContentBackground(.hidden) + .padding(6) + .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 6)) + HStack { + Button { + Task { await model.run() } + } label: { + Label("Ask", systemImage: "play.fill") + } + .keyboardShortcut(.return, modifiers: .command) + .buttonStyle(.borderedProminent) + .disabled(model.isRunning || model.set.questions.isEmpty) + if model.isRunning { ProgressView().controlSize(.small) } + Spacer() + Text(model.activeBackendLabels.joined(separator: " · ")).font(.jCaption).foregroundStyle(.secondary) + } + + if model.results.isEmpty { + HowItWorks() + .padding(.top, 12) + } else { + let shown = model.results.map(model.display) + if let first = shown.first(where: { $0.result != nil })?.result, first.input != model.input { + Text("Results are for the previous input.").font(.jCaption).foregroundStyle(.secondary) + } + RunSummary(runs: shown) + ForEach(questionIDs, id: \.self) { qid in + QuestionCompareCard(entries: shown.map { run in + (run.label, run.result?.results.first { $0.question.id == qid }, run.error) + }) + } + } + } + .padding() + } + } + + /// Question ids in the order the first successful backend answered them. + private var questionIDs: [UUID] { + model.results.first { $0.result != nil }?.result?.results.map(\.question.id) ?? [] + } + + // MARK: Batch + + private var batch: some View { + VStack(alignment: .leading, spacing: 10) { + Text("Inputs, one per line").font(.jCallout.weight(.medium)) + TextEditor(text: $model.batchText) + .font(.jCallout) + .frame(minHeight: 90, maxHeight: 160) + .scrollContentBackground(.hidden) + .padding(6) + .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 6)) + HStack { + if model.batchProgress != nil { + Button("Stop") { model.stopBatch() } + } else { + Button { + model.runBatch() + } label: { + Label("Ask all \(model.batchInputs.count)", systemImage: "play.fill") + } + .buttonStyle(.borderedProminent) + .disabled(model.isRunning || model.batchInputs.isEmpty || model.set.questions.isEmpty) + } + if let p = model.batchProgress { + ProgressView(value: Double(p.done), total: Double(max(p.total, 1))).frame(width: 140) + Text("\(p.done)/\(p.total)").font(.jCaption).foregroundStyle(.secondary) + } + Spacer() + if model.markedCount > 0 { + Text("\(model.markedCount) correct answers marked").font(.jCaption).foregroundStyle(.secondary) + } + if model.applyCalibration, model.calibration != nil, model.calibrationMismatch == nil { + Label("calibrated", systemImage: "dial.medium").font(.jCaption).foregroundStyle(.tint) + } + Menu { + Button("Import answer set (CSV)…") { model.importAnswerSet() } + Button("Export answer set (CSV)…") { model.exportAnswerSet() } + } label: { + Label("Answer set", systemImage: "checklist") + } + .fixedSize() + .help("Load inputs with their correct answers from a CSV (input column, then one column per question), or save yours") + Button("Calibrate…") { showCalibration = true } + .help("Mark correct answers in the grid, then fit a calibration for the local model") + Menu { + if let path = model.resultsCSVPath { + Button("Append to \(URL(fileURLWithPath: path).lastPathComponent)") { model.appendResults(choose: false) } + } + Button("Append to another CSV…") { model.appendResults(choose: true) } + } label: { + Label("Results CSV", systemImage: "tablecells") + } primaryAction: { + model.appendResults(choose: model.resultsCSVPath == nil) + } + .fixedSize() + .disabled(model.batchRows.isEmpty || model.batchProgress != nil) + .help("Append this run to a results CSV: one row per input × question × backend, with the model, your marked answer, the answer, confidence and timing. Later runs append, so models line up in one spreadsheet.") + } + if let a = model.lastAppend { + Text("Appended \(a.rows) rows to \(a.file).").font(.jCaption).foregroundStyle(.secondary) + } + BatchGrid(model: model) + } + .padding() + .sheet(isPresented: $showCalibration) { CalibrationSheet(model: model) } + } +} + +/// Per backend: time, tokens/cost, or the error. +struct RunSummary: View { + let runs: [BackendRun] + + var body: some View { + VStack(alignment: .leading, spacing: 2) { + ForEach(runs) { run in + HStack(spacing: 6) { + Text(run.label).fontWeight(.medium) + if let r = run.result { + Text("\(Int(r.milliseconds)) ms").foregroundStyle(.secondary) + if let u = r.usage { Text("· \(u)").foregroundStyle(.secondary) } + if r.sharedPrefixTokens > 0 { Text("· \(r.sharedPrefixTokens) shared prompt tokens").foregroundStyle(.secondary) } + } else if let e = run.error { + Text(e).foregroundStyle(.orange).lineLimit(2) + } + } + } + } + .font(.jCaption) + } +} + +/// One question, answered by each backend. +struct QuestionCompareCard: View { + let entries: [(label: String, result: QuestionResult?, error: String?)] + @State private var showPrompt: String? + + private var question: EditableQuestion? { entries.compactMap(\.result).first?.question } + + var body: some View { + GroupBox { + VStack(alignment: .leading, spacing: 8) { + if let q = question { + HStack(alignment: .firstTextBaseline) { + Text(q.name).font(.system(size: 14, design: .monospaced).weight(.semibold)) + Text(q.kind.displayName).font(.jCaption).foregroundStyle(.secondary) + Spacer() + if agreement == false { + Label("Backends disagree", systemImage: "arrow.triangle.branch") + .font(.jCaption).foregroundStyle(.orange) + } + } + Text(q.text).font(.jCaption).foregroundStyle(.secondary) + } + ForEach(entries.indices, id: \.self) { i in + let e = entries[i] + if let r = e.result { + BackendAnswer(label: e.label, result: r, showPrompt: showPrompt == e.label) { + showPrompt = showPrompt == e.label ? nil : e.label + } + } else if let err = e.error { + Text("\(e.label): \(err)").font(.jCaption).foregroundStyle(.orange) + } + if i < entries.count - 1 { Divider() } + } + } + .padding(4) + } + } + + /// nil when fewer than two backends answered. + private var agreement: Bool? { + let answers = entries.compactMap(\.result).map(\.answer) + return answers.count < 2 ? nil : Set(answers).count == 1 + } +} + +/// One backend's bars for one question. +struct BackendAnswer: View { + let label: String + let result: QuestionResult + let showPrompt: Bool + let togglePrompt: () -> Void + + var body: some View { + VStack(alignment: .leading, spacing: 4) { + HStack(alignment: .firstTextBaseline) { + Text(label).font(.jCaption.weight(.semibold)) + Text(fidelityText).font(.jCaption2).foregroundStyle(.secondary) + Spacer() + Text(result.answer).font(.jHeadline) + Text(percent(result.confidence)) + .font(.jHeadline.monospacedDigit()) + .foregroundStyle(confidenceColor(result.confidence)) + } + ForEach(result.keys.indices, id: \.self) { i in + HStack(spacing: 8) { + Text(result.labels[i]) + .font(.system(size: 12, design: .monospaced).weight(.bold)) + .frame(width: 28, alignment: .leading) + Text(optionText(i)) + .font(.jCaption) + .lineLimit(1) + .frame(width: 175, alignment: .leading) + GeometryReader { g in + ZStack(alignment: .leading) { + RoundedRectangle(cornerRadius: 3).fill(.quaternary) + RoundedRectangle(cornerRadius: 3) + .fill(i == result.topIndex ? AnyShapeStyle(.tint) : AnyShapeStyle(.secondary.opacity(0.5))) + .frame(width: max(2, g.size.width * result.probabilities[i])) + } + } + .frame(height: 8) + Text(percent(result.probabilities[i])) + .font(.jCaption.monospacedDigit()) + .frame(width: 44, alignment: .trailing) + } + } + HStack { + if result.labelMass < 0.9 { + Label("Off format: only \(percent(result.labelMass)) of the probability was on the allowed answers.", + systemImage: "exclamationmark.triangle") + .foregroundStyle(.orange) + } + Spacer() + Button(showPrompt ? "Hide" : (label.hasPrefix("Local") ? "Show prompt" : "Show request"), action: togglePrompt) + .buttonStyle(.link) + } + .font(.jCaption) + if showPrompt { + Text(result.prompt) + .font(.system(size: 12, design: .monospaced)) + .textSelection(.enabled) + .padding(8) + .frame(maxWidth: .infinity, alignment: .leading) + .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 6)) + } + } + } + + private var fidelityText: String { + switch result.fidelity { + case .native: "decision model" + case .tokenScored(let calibrated): calibrated ? "token-scored, calibrated" : "token-scored" + @unknown default: "" + } + } + + private func optionText(_ i: Int) -> String { + switch result.question.kind { + case .noul: "" + case .choice: result.keys[i] + case .score: result.descriptions[i] + } + } +} + +/// Inputs × questions; each cell has one line per backend, and a "mark…" menu to record the +/// correct answer (for calibration and per-backend accuracy). +struct BatchGrid: View { + let model: AppModel + + var body: some View { + let rows = model.batchRows.map { BatchRow(input: $0.input, runs: $0.runs.map(model.display)) } + if let first = rows.first, let template = first.runs.first(where: { $0.result != nil })?.result { + Text("\(template.results.count) questions × \(rows.count) inputs\(first.runs.count > 1 ? " × \(first.runs.count) backends" : "") · one column per question; scroll sideways if they don't all fit, or hide the questions panel (toolbar) for more room") + .font(.jCaption).foregroundStyle(.secondary) + ScrollView([.horizontal, .vertical]) { + Grid(alignment: .leading, horizontalSpacing: 10, verticalSpacing: 6) { + GridRow { + Text("Input").font(.jCaption.weight(.semibold)) + ForEach(template.results) { r in + QuestionHeader(result: r) + } + } + if first.runs.count > 1 { + GridRow { + Text(first.runs.enumerated().map { "\(tag($0.offset)) \($0.element.label)" }.joined(separator: " ")) + .font(.jCaption2).foregroundStyle(.secondary) + .gridCellColumns(template.results.count + 1) + } + } + Divider() + ForEach(rows) { row in + GridRow(alignment: .top) { + Text(row.input) + .font(.jCaption) + .lineLimit(4) + .frame(width: 220, alignment: .leading) + .help(row.input) + ForEach(template.results.indices, id: \.self) { qi in + let q = template.results[qi] + let mark = model.label(input: row.input, question: q.question.name) + VStack(alignment: .leading, spacing: 2) { + ForEach(row.runs.indices, id: \.self) { bi in + cell(row.runs[bi], qi, tag: row.runs.count > 1 ? tag(bi) : nil, mark: mark) + } + markMenu(row.input, q, mark) + } + .padding(3) + .frame(minWidth: 150, alignment: .leading) + .background(disagrees(row, qi) ? Color.orange.opacity(0.08) : .clear, + in: RoundedRectangle(cornerRadius: 4)) + } + } + } + } + .padding(.vertical, 4) + } + .scrollIndicators(.visible) + } else if let err = rows.first?.runs.first?.error { + Text(err).foregroundStyle(.orange).frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading) + } else { + Text("Batch asks the same questions about every input above, like a spreadsheet: one row per input, one column per question, each cell an answer. Hover a cell for every option's probability. Use “mark…” under a cell to record the correct answer: each backend then shows ✓ or ✗, and Calibrate… fits the local model's confidence to your marks.") + .foregroundStyle(.secondary) + .frame(maxWidth: .infinity, maxHeight: .infinity, alignment: .topLeading) + } + } + + private func tag(_ i: Int) -> String { ["①", "②", "③", "④"][min(i, 3)] } + + @ViewBuilder + private func cell(_ run: BackendRun, _ qi: Int, tag: String?, mark: String?) -> some View { + if let results = run.result?.results, results.indices.contains(qi) { + let r = results[qi] + let verdict = mark.map { $0 == r.answer ? "✓ " : "✗ " } ?? "" + Text("\(tag.map { $0 + " " } ?? "")\(verdict)\(r.answer) \(percent(r.confidence))") + .font(.jCaption.monospacedDigit()) + .padding(.horizontal, 5) + .padding(.vertical, 1) + .background(confidenceColor(r.confidence).opacity(0.18), in: RoundedRectangle(cornerRadius: 4)) + .help("\(run.label)\(r.fidelity == .tokenScored(calibrated: true) ? " (calibrated)" : "")\n" + + zip(r.keys, r.probabilities).map { "\($0): \(percent($1))" }.joined(separator: "\n")) + } else { + Text("\(tag.map { $0 + " " } ?? "")error").font(.jCaption).foregroundStyle(.orange).help(run.error ?? "") + } + } + + private func markMenu(_ input: String, _ q: QuestionResult, _ mark: String?) -> some View { + Menu { + ForEach(q.keys, id: \.self) { key in + Button(key) { model.setLabel(key, input: input, question: q.question.name) } + } + if mark != nil { + Divider() + Button("Clear") { model.setLabel(nil, input: input, question: q.question.name) } + } + } label: { + Text(mark.map { "correct: \($0)" } ?? "mark…") + .font(.jCaption2) + .foregroundStyle(mark == nil ? .secondary : .primary) + } + .menuStyle(.borderlessButton) + .menuIndicator(.hidden) + .fixedSize() + .help("The correct answer for this input") + } + + private func disagrees(_ row: BatchRow, _ qi: Int) -> Bool { + let answers = row.runs.compactMap { $0.result?.results }.filter { $0.indices.contains(qi) }.map { $0[qi].answer } + return answers.count > 1 && Set(answers).count > 1 + } +} + +/// A batch-grid column header: the question's name, then the answers it allows. Hover for the +/// question text and each option's description. +struct QuestionHeader: View { + let result: QuestionResult + + var body: some View { + VStack(alignment: .leading, spacing: 1) { + Text(result.question.name).font(.system(size: 12, design: .monospaced).weight(.semibold)) + Text(options).font(.jCaption).foregroundStyle(.secondary).lineLimit(2) + } + .frame(maxWidth: 200, alignment: .leading) + .help(detail) + } + + private var options: String { + switch result.question.kind { + case .noul: "Yes / No" + case .choice: result.keys.joined(separator: " / ") + case .score: zip(result.keys, result.descriptions).map { "\($0) \($1)" }.joined(separator: " · ") + } + } + + private var detail: String { + var s = "\(result.question.text)\n\n\(result.question.kind.displayName)" + if result.question.kind != .noul { + s += "\n" + zip(result.keys, result.descriptions) + .map { k, d in d.isEmpty || d == k ? "• \(k)" : "• \(k): \(d)" }.joined(separator: "\n") + } + return s + } +} + +func percent(_ p: Double) -> String { "\(Int((p * 100).rounded()))%" } + +func confidenceColor(_ p: Double) -> Color { + p >= 0.9 ? .green : p >= 0.6 ? .yellow : .orange +} + +/// Shown before the first Ask: what a decider does, in three lines. +struct HowItWorks: View { + var body: some View { + VStack(alignment: .leading, spacing: 10) { + Text("How this works").font(.jCallout.weight(.semibold)) + row("1", "One input", "a message, a query, a review: whatever your app needs to decide about.") + row("2", "A few fixed questions", "on the left. Each allows only certain answers: yes / no, one of several options, or a point on a scale.") + row("3", "One answer per question", "with how sure the model is. No text is written; your app acts on the answers in code.") + Text("Try Examples → Customer support, then click a sample message above the input box. Batch runs the same questions over many inputs at once.") + .font(.jCaption) + .foregroundStyle(.secondary) + .padding(.top, 4) + } + .padding(12) + .frame(maxWidth: .infinity, alignment: .leading) + .background(.quaternary.opacity(0.4), in: RoundedRectangle(cornerRadius: 8)) + } + + private func row(_ n: String, _ title: String, _ detail: String) -> some View { + HStack(alignment: .firstTextBaseline, spacing: 8) { + Text(n) + .font(.jCaption.weight(.bold)) + .frame(width: 18, height: 18) + .background(.tint.opacity(0.18), in: Circle()) + (Text(title).fontWeight(.medium) + Text(": ") + Text(detail).foregroundStyle(.secondary)) + .font(.jCallout) + } + } +} diff --git a/examples/jevdk/Sources/OpenJevKit/EditorModel.swift b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift new file mode 100644 index 0000000..0c2e139 --- /dev/null +++ b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift @@ -0,0 +1,254 @@ +import Foundation +import LocalLMLabSDKCore + +// What JevDK's editor edits and its results show. The decision API itself is the SDK's +// (`DecisionQuestion`, `lab.decide`, `Decision`); these types add what a playground needs on top: +// stable identities for editing, a saved-file format, and per-answer display rows. + +/// What kind of answer a question takes. Jev's names. +public enum QuestionKind: String, Codable, CaseIterable, Sendable { + /// True or false; the answer is the probability it's true. + case noul + /// One of several named options. + case choice + /// An ordered scale, low to high. Small local models do worst on these. + case score + + public var displayName: String { + switch self { + case .noul: "Noul (yes / no)" + case .choice: "Choice" + case .score: "Score" + } + } + + // Files saved before the rename used yesNo / choice / scale. + public init(from decoder: Swift.Decoder) throws { + let raw = try decoder.singleValueContainer().decode(String.self) + switch raw { + case "noul", "yesNo": self = .noul + case "choice": self = .choice + case "score", "scale": self = .score + default: + throw DecodingError.dataCorrupted(.init(codingPath: decoder.codingPath, debugDescription: "unknown question kind \(raw)")) + } + } +} + +/// One option of a choice question, or one level of a score question. +public struct AnswerOption: Codable, Hashable, Identifiable, Sendable { + public var id: UUID + /// What the app reads back (choice only; a score's levels are `0`, `1`, …). + public var key: String + /// What the model reads next to the option. + public var description: String + + public init(id: UUID = UUID(), key: String, description: String) { + self.id = id; self.key = key; self.description = description + } + + public init(from decoder: Swift.Decoder) throws { + let c = try decoder.container(keyedBy: CodingKeys.self) + id = try c.decodeIfPresent(UUID.self, forKey: .id) ?? UUID() + key = try c.decode(String.self, forKey: .key) + description = try c.decodeIfPresent(String.self, forKey: .description) ?? "" + } +} + +/// A question as the editor holds it. +public struct EditableQuestion: Codable, Hashable, Identifiable, Sendable { + public var id: UUID + /// The id the app reads the answer by, e.g. `needsLiveData`. + public var name: String + public var kind: QuestionKind + /// The question as the model reads it (Jev's `instructions`). + public var text: String + /// Choice options or score levels. Ignored for noul. + public var options: [AnswerOption] + + public init(id: UUID = UUID(), name: String, kind: QuestionKind, text: String, options: [AnswerOption] = []) { + self.id = id; self.name = name; self.kind = kind; self.text = text; self.options = options + } + + public init(from decoder: Swift.Decoder) throws { + let c = try decoder.container(keyedBy: CodingKeys.self) + id = try c.decodeIfPresent(UUID.self, forKey: .id) ?? UUID() + name = try c.decode(String.self, forKey: .name) + kind = try c.decode(QuestionKind.self, forKey: .kind) + text = try c.decode(String.self, forKey: .text) + options = try c.decodeIfPresent([AnswerOption].self, forKey: .options) ?? [] + } + + /// The SDK question this becomes. + public var sdkQuestion: DecisionQuestion { + switch kind { + case .noul: + return .noul(name, text) + case .choice: + return DecisionQuestion(id: name, instructions: text, kind: .choice(criteria: options.map { + .init($0.key.isEmpty ? $0.description : $0.key, $0.description.isEmpty ? nil : $0.description) + })) + case .score: + return .score(name, text, criteria: options.map(\.description)) + } + } + + /// Editor-side problems, shown under the card. `lab.decide` validates again before sending. + public var problems: [String] { + var p: [String] = [] + if name.trimmingCharacters(in: .whitespaces).isEmpty { p.append("needs a name") } + if text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty { p.append("question text is empty") } + if kind != .noul { + if options.count < 2 { p.append("needs at least 2 options") } + if options.contains(where: { $0.description.trimmingCharacters(in: .whitespaces).isEmpty }) { + p.append("an option has no description") + } + if kind == .choice { + let keys = options.map { $0.key.isEmpty ? $0.description : $0.key } + if Set(keys).count != keys.count { p.append("option keys must be unique") } + } + } + return p + } +} + +/// A named set of questions plus the local wrapper they're tuned with. What JevDK saves. +public struct QuestionSet: Codable, Hashable, Sendable { + /// The SDK's `OpenJevWrapper.default` (chosen with the eval, D1b), mirrored here so this + /// module needs only Core. JevDK checks they match at launch. + public static let defaultSystem = "You answer questions about a piece of text. Read the text, then answer the question about it. Reply with only the label of the correct answer." + public static let defaultInputLabel = "Text" + + public var name: String + /// Local OpenJev's wrapper instructions. Hosted Jev never sees them. + public var system: String + /// What the input is called in the local prompt ("Input", "Request", …). + public var inputLabel: String + public var questions: [EditableQuestion] + + public init(name: String, system: String = QuestionSet.defaultSystem, inputLabel: String = QuestionSet.defaultInputLabel, + questions: [EditableQuestion]) { + self.name = name; self.system = system; self.inputLabel = inputLabel; self.questions = questions + } + + public init(from decoder: Swift.Decoder) throws { + let c = try decoder.container(keyedBy: CodingKeys.self) + name = try c.decode(String.self, forKey: .name) + system = try c.decodeIfPresent(String.self, forKey: .system) ?? QuestionSet.defaultSystem + inputLabel = try c.decodeIfPresent(String.self, forKey: .inputLabel) ?? QuestionSet.defaultInputLabel + questions = try c.decode([EditableQuestion].self, forKey: .questions) + } + +} + +// MARK: - Display + +/// What a local run measured per question beyond the `Decision` (from the SDK's +/// `OpenJevDiagnostics`). +public struct QuestionDiagnostics: Sendable { + /// The single-token labels the model was asked for. + public let labels: [String] + /// The exact text the model received. + public let prompt: String + /// Probability on the allowed labels before normalising. + public let labelMass: Double + public let milliseconds: Double + + public init(labels: [String], prompt: String, labelMass: Double, milliseconds: Double) { + self.labels = labels; self.prompt = prompt; self.labelMass = labelMass; self.milliseconds = milliseconds + } +} + +/// One question's answer as a row of bars: the SDK `Decision` plus, for local runs, the +/// OpenJev diagnostics the SDK API doesn't carry. +public struct QuestionResult: Identifiable, Sendable { + public var id: UUID { question.id } + public let question: EditableQuestion + public let keys: [String] + /// The labels the local model was asked for (`A`, `0`, `Yes`); keys for hosted runs. + public let labels: [String] + public let descriptions: [String] + /// As shown: calibrated when JevDK is applying a calibration, raw otherwise. + public let probabilities: [Double] + /// As the backend returned them, before any calibration JevDK applies. What calibration is + /// fitted on. + public let rawProbabilities: [Double] + /// The answer's confidence as the backend reports it. + public let confidence: Double + /// Local only: probability on the allowed labels before normalising. 1 for hosted runs. + public let labelMass: Double + /// Local: the exact prompt. Hosted: the exact request JSON. + public let prompt: String + public let milliseconds: Double + public let fidelity: DecisionFidelity + + public var topIndex: Int { probabilities.indices.max { probabilities[$0] < probabilities[$1] } ?? 0 } + public var answer: String { keys[topIndex] } + + /// This answer with `probabilities` shown instead (a calibration applied to the raw ones). + public func showing(_ probabilities: [Double], calibrated: Bool) -> QuestionResult { + let top = probabilities.indices.max { probabilities[$0] < probabilities[$1] } ?? 0 + return QuestionResult(question: question, keys: keys, labels: labels, descriptions: descriptions, + probabilities: probabilities, rawProbabilities: rawProbabilities, + confidence: probabilities.isEmpty ? 0 : probabilities[top], labelMass: labelMass, + prompt: prompt, milliseconds: milliseconds, + fidelity: .tokenScored(calibrated: calibrated)) + } + + /// Build display rows from an SDK decision. `diagnostics` (local) supplies labels, prompts and + /// label mass; `prompt` is used for every row otherwise (hosted: the request JSON). + public static func rows(for questions: [EditableQuestion], decision d: Decision, + diagnostics: [String: QuestionDiagnostics] = [:], prompt: String = "") -> [QuestionResult] { + let perQuestionMS = Double(d.latency.components.seconds) * 1000 / Double(max(questions.count, 1)) + + Double(d.latency.components.attoseconds) / 1e15 / Double(max(questions.count, 1)) + return questions.compactMap { q in + let diag = diagnostics[q.name] + let keys: [String], descs: [String], probs: [Double], conf: Double + switch d.answers[q.name] { + case .noul(let p)?: + keys = ["Yes", "No"]; descs = ["", ""]; probs = [p, 1 - p]; conf = max(p, 1 - p) + case .choice(let c)?: + let sq = q.sdkQuestion + guard case .choice(let criteria) = sq.kind else { return nil } + keys = criteria.map(\.key); descs = criteria.map { $0.description ?? "" } + probs = keys.map { c.probabilities[$0] ?? 0 }; conf = c.confidence + case .score(let s)?: + keys = s.legend.indices.map(String.init); descs = s.legend; probs = s.probabilities; conf = s.confidence + default: + return nil + } + return QuestionResult( + question: q, keys: keys, labels: diag?.labels ?? keys, descriptions: descs, probabilities: probs, + rawProbabilities: probs, confidence: conf, labelMass: diag?.labelMass ?? 1, prompt: diag?.prompt ?? prompt, + milliseconds: diag?.milliseconds ?? perQuestionMS, fidelity: d.fidelity) + } + } +} + +/// All the questions' answers for one input from one backend. +public struct DecisionResult: Sendable { + public let input: String + public let results: [QuestionResult] + /// Local only: prompt tokens shared by every question and computed once. + public let sharedPrefixTokens: Int + public let milliseconds: Double + public let backend: String + /// Hosted only: tokens and cost. + public var usage: String? + + public init(input: String, results: [QuestionResult], sharedPrefixTokens: Int, milliseconds: Double, + backend: String, usage: String? = nil) { + self.input = input; self.results = results; self.sharedPrefixTokens = sharedPrefixTokens + self.milliseconds = milliseconds; self.backend = backend; self.usage = usage + } +} + +extension DecisionUsage { + /// "532 in / 105 out tokens · $0.000022" + public var summary: String? { + var parts: [String] = [] + if let i = inputTokens, let o = outputTokens { parts.append("\(i) in / \(o) out tokens") } + if let c = cost { parts.append(String(format: "$%.6f", c)) } + return parts.isEmpty ? nil : parts.joined(separator: " · ") + } +} diff --git a/examples/jevdk/Sources/OpenJevKit/ModelCatalog.swift b/examples/jevdk/Sources/OpenJevKit/ModelCatalog.swift new file mode 100644 index 0000000..147881f --- /dev/null +++ b/examples/jevdk/Sources/OpenJevKit/ModelCatalog.swift @@ -0,0 +1,28 @@ +import Foundation + +/// What's in the local Hugging Face cache, by repo id. The SDK only counts a model it downloaded +/// and verified itself (`MLXModelProvider.installed`); this lists the rest, so JevDK can offer to +/// verify a copy fetched by another tool instead of downloading it again. +public enum ModelCatalog { + /// `~/.cache/huggingface/hub`, or `HF_HUB_CACHE` / `HF_HOME/hub` when set. + public static var cacheRoot: URL { + let env = ProcessInfo.processInfo.environment + if let hub = env["HF_HUB_CACHE"] { return URL(fileURLWithPath: hub) } + if let home = env["HF_HOME"] { return URL(fileURLWithPath: home).appendingPathComponent("hub") } + return FileManager.default.homeDirectoryForCurrentUser.appendingPathComponent(".cache/huggingface/hub") + } + + /// Repo ids with at least one snapshot holding a `config.json` and safetensors weights. + public static func cachedRepoIDs() -> [String] { + let fm = FileManager.default + guard let repos = try? fm.contentsOfDirectory(atPath: cacheRoot.path) else { return [] } + return repos.filter { $0.hasPrefix("models--") }.compactMap { repo in + let snapshots = cacheRoot.appendingPathComponent(repo).appendingPathComponent("snapshots") + let hasWeights = ((try? fm.contentsOfDirectory(atPath: snapshots.path)) ?? []).contains { rev in + let files = (try? fm.contentsOfDirectory(atPath: snapshots.appendingPathComponent(rev).path)) ?? [] + return files.contains("config.json") && files.contains { $0.hasSuffix(".safetensors") } + } + return hasWeights ? String(repo.dropFirst("models--".count)).replacingOccurrences(of: "--", with: "/") : nil + }.sorted() + } +} From 860ec1b6ca378cc3699a43c4adef60d6eb9e2867 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 19:42:50 -0700 Subject: [PATCH 27/73] examples: add Xcode project for jevdk XcodeGen project.yml + generated JevDK.xcodeproj: ad-hoc signed, not sandboxed, so it shares the Hugging Face cache and Keychain keys with the swift build binary. Package.swift vends the three SDK binaries as library products for it. Co-Authored-By: Claude Opus 5.5 --- examples/README.md | 2 +- .../jevdk/JevDK.xcodeproj/project.pbxproj | 407 ++++++++++++++++++ .../contents.xcworkspacedata | 7 + .../xcshareddata/xcschemes/JevDK.xcscheme | 91 ++++ examples/jevdk/Package.swift | 5 + examples/jevdk/project.yml | 68 +++ examples/jevdk/xcodeproj/JevDK-Info.plist | 30 ++ 7 files changed, 609 insertions(+), 1 deletion(-) create mode 100644 examples/jevdk/JevDK.xcodeproj/project.pbxproj create mode 100644 examples/jevdk/JevDK.xcodeproj/project.xcworkspace/contents.xcworkspacedata create mode 100644 examples/jevdk/JevDK.xcodeproj/xcshareddata/xcschemes/JevDK.xcscheme create mode 100644 examples/jevdk/project.yml create mode 100644 examples/jevdk/xcodeproj/JevDK-Info.plist diff --git a/examples/README.md b/examples/README.md index 6d17496..062db36 100644 --- a/examples/README.md +++ b/examples/README.md @@ -72,7 +72,7 @@ build against the latest stable release. No environment variable is needed for e | `mlx-control-room` | SwiftUI + App Sandbox (network only) | ✅ via the committed `.xcodeproj` — **Automatic** signing (as above); it keeps no security-scoped bookmark, so **Sign to Run Locally** works too | **any** identity — `packaging/build-and-sign.sh` | | `components-updates-demo` | SwiftUI, no system permissions, simulated data | ✅ via `Package.swift` / `swift run` (no `.xcodeproj`) | — (no `packaging/`) | | `workspace-buddy`, `workspace-buddy-local`, `aiql` | SwiftUI + App Sandbox | ✅ via the committed `.xcodeproj` — **Automatic** signing (as above); a stable team identity is what lets the security-scoped bookmark survive a rebuild. Bare `swift run` is compile-only. | same as `plate-today` | -| `jevdk` | SwiftUI, no system permissions, no sandbox; hosted deciders optional (Featherless demo needs no key) | ✅ via `Package.swift` / `swift run -c release JevDK` (no `.xcodeproj`) | — (no `packaging/`) | +| `jevdk` | SwiftUI, no system permissions, no sandbox; hosted deciders optional (Featherless demo needs no key) | ✅ via the committed `.xcodeproj` — ad-hoc signed (*Sign to Run Locally*), not sandboxed, no account needed; or `swift build -c release` and run `.build/release/JevDK` | — (no `packaging/`) | | `mcp-chat` | SwiftUI + App Sandbox (network) | Compile-only via `Package.swift` / `swift build` (no `.xcodeproj`) | **any** identity, or **none** (`APP_IDENTITY=-`) — `packaging/build-and-sign.sh` | "Apple Development" = the free identity Xcode creates once you add any Apple ID under diff --git a/examples/jevdk/JevDK.xcodeproj/project.pbxproj b/examples/jevdk/JevDK.xcodeproj/project.pbxproj new file mode 100644 index 0000000..62c6e97 --- /dev/null +++ b/examples/jevdk/JevDK.xcodeproj/project.pbxproj @@ -0,0 +1,407 @@ +// !$*UTF8*$! +{ + archiveVersion = 1; + classes = { + }; + objectVersion = 77; + objects = { + +/* Begin PBXBuildFile section */ + 01C38D4ACEE528CDA8C678B0 /* AppModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2E61A2E527AD1DCB8F14B9C3 /* AppModel.swift */; }; + 18CD55EBA24EC88C2B7F802F /* Presets.swift in Sources */ = {isa = PBXBuildFile; fileRef = 9ED714827E3A9F073BFE1E5D /* Presets.swift */; }; + 41CDC20EF81297CAE99A72F9 /* ModelsSheet.swift in Sources */ = {isa = PBXBuildFile; fileRef = E4D7E0C08FF14EDD165B3304 /* ModelsSheet.swift */; }; + 545585A403260A8CD9BB82AB /* CalibrationSheet.swift in Sources */ = {isa = PBXBuildFile; fileRef = CB669C7255911EF7E56649DC /* CalibrationSheet.swift */; }; + 54EAB33CF8CCFD4279D2FB6F /* Keychain.swift in Sources */ = {isa = PBXBuildFile; fileRef = 3DCFF18378FD51FAF9D3CA71 /* Keychain.swift */; }; + 6C032E883ED0732D4FA1E543 /* HostedBackends.swift in Sources */ = {isa = PBXBuildFile; fileRef = 0AED34770F4AED1DBB9A1AB2 /* HostedBackends.swift */; }; + 797DFE0A2A40F77BBD987A35 /* EditorView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 14B240BCB75D0E2D2DDCD444 /* EditorView.swift */; }; + 87342ABCE02C15802B9E51B8 /* LocalLMLabSDKInference in Frameworks */ = {isa = PBXBuildFile; productRef = B3CB5FCBA59A48CE738CD765 /* LocalLMLabSDKInference */; }; + 8BABC31DA33A9AFFC2F0DA55 /* Calibration.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8F632766752D73CE9BCD20C9 /* Calibration.swift */; }; + B3F706C436621FC2C4E031C5 /* Fonts.swift in Sources */ = {isa = PBXBuildFile; fileRef = 163802A0E4C033A476FD0751 /* Fonts.swift */; }; + BA54F7E66BD3069DA73E272D /* LocalLMLabSDKRemote in Frameworks */ = {isa = PBXBuildFile; productRef = 08F14639708048E2EF3B8497 /* LocalLMLabSDKRemote */; }; + C2BE5CB8BA2C66DE7D06050D /* OpenJevKit in Frameworks */ = {isa = PBXBuildFile; productRef = A76F927BDDBECBFD1D95BACB /* OpenJevKit */; }; + DDD7A7D0558AB7E447228BDC /* ResultsView.swift in Sources */ = {isa = PBXBuildFile; fileRef = E6051C9473DA4BE820910F6A /* ResultsView.swift */; }; + E7344F9A7EC5BD75E35A2BDE /* LocalLMLabSDKCore in Frameworks */ = {isa = PBXBuildFile; productRef = 4AD390EEAA615C4C5D4C5418 /* LocalLMLabSDKCore */; }; + F141C2660F85B1C1CA748204 /* ResultsCSV.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1745A36CAECF9C9049D439F9 /* ResultsCSV.swift */; }; + F226A1630946764D74AA1773 /* JevDKApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = A2D4489C0C849F6196719ED0 /* JevDKApp.swift */; }; +/* End PBXBuildFile section */ + +/* Begin PBXFileReference section */ + 0AED34770F4AED1DBB9A1AB2 /* HostedBackends.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = HostedBackends.swift; sourceTree = ""; }; + 14B240BCB75D0E2D2DDCD444 /* EditorView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = EditorView.swift; sourceTree = ""; }; + 163802A0E4C033A476FD0751 /* Fonts.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Fonts.swift; sourceTree = ""; }; + 1745A36CAECF9C9049D439F9 /* ResultsCSV.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ResultsCSV.swift; sourceTree = ""; }; + 2E61A2E527AD1DCB8F14B9C3 /* AppModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppModel.swift; sourceTree = ""; }; + 3DCFF18378FD51FAF9D3CA71 /* Keychain.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Keychain.swift; sourceTree = ""; }; + 69F4D6841A944F1883A47736 /* JevDK.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = JevDK.app; sourceTree = BUILT_PRODUCTS_DIR; }; + 8F632766752D73CE9BCD20C9 /* Calibration.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Calibration.swift; sourceTree = ""; }; + 94BDB6CCEB9C69E8CDD41E8F /* jevdk */ = {isa = PBXFileReference; lastKnownFileType = folder; name = jevdk; path = .; sourceTree = SOURCE_ROOT; }; + 9ED714827E3A9F073BFE1E5D /* Presets.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Presets.swift; sourceTree = ""; }; + A2D4489C0C849F6196719ED0 /* JevDKApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = JevDKApp.swift; sourceTree = ""; }; + CB669C7255911EF7E56649DC /* CalibrationSheet.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = CalibrationSheet.swift; sourceTree = ""; }; + E4D7E0C08FF14EDD165B3304 /* ModelsSheet.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ModelsSheet.swift; sourceTree = ""; }; + E6051C9473DA4BE820910F6A /* ResultsView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ResultsView.swift; sourceTree = ""; }; +/* End PBXFileReference section */ + +/* Begin PBXFrameworksBuildPhase section */ + 619937F107583FC0C4C18B60 /* Frameworks */ = { + isa = PBXFrameworksBuildPhase; + buildActionMask = 2147483647; + files = ( + C2BE5CB8BA2C66DE7D06050D /* OpenJevKit in Frameworks */, + E7344F9A7EC5BD75E35A2BDE /* LocalLMLabSDKCore in Frameworks */, + BA54F7E66BD3069DA73E272D /* LocalLMLabSDKRemote in Frameworks */, + 87342ABCE02C15802B9E51B8 /* LocalLMLabSDKInference in Frameworks */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXFrameworksBuildPhase section */ + +/* Begin PBXGroup section */ + 4A134F6EE83A56CA4F797A97 /* JevDK */ = { + isa = PBXGroup; + children = ( + 2E61A2E527AD1DCB8F14B9C3 /* AppModel.swift */, + 8F632766752D73CE9BCD20C9 /* Calibration.swift */, + CB669C7255911EF7E56649DC /* CalibrationSheet.swift */, + 14B240BCB75D0E2D2DDCD444 /* EditorView.swift */, + 163802A0E4C033A476FD0751 /* Fonts.swift */, + 0AED34770F4AED1DBB9A1AB2 /* HostedBackends.swift */, + A2D4489C0C849F6196719ED0 /* JevDKApp.swift */, + 3DCFF18378FD51FAF9D3CA71 /* Keychain.swift */, + E4D7E0C08FF14EDD165B3304 /* ModelsSheet.swift */, + 9ED714827E3A9F073BFE1E5D /* Presets.swift */, + 1745A36CAECF9C9049D439F9 /* ResultsCSV.swift */, + E6051C9473DA4BE820910F6A /* ResultsView.swift */, + ); + path = JevDK; + sourceTree = ""; + }; + 6FE9F03C6AB6D0B3A2D996AB /* Sources */ = { + isa = PBXGroup; + children = ( + 4A134F6EE83A56CA4F797A97 /* JevDK */, + ); + path = Sources; + sourceTree = ""; + }; + A64CD306E30FE5B113DA1DAF = { + isa = PBXGroup; + children = ( + D36EC97BBA78359044974DDA /* Packages */, + 6FE9F03C6AB6D0B3A2D996AB /* Sources */, + DD4AF31F75D7D5BA45059FDA /* Products */, + ); + sourceTree = ""; + }; + D36EC97BBA78359044974DDA /* Packages */ = { + isa = PBXGroup; + children = ( + 94BDB6CCEB9C69E8CDD41E8F /* jevdk */, + ); + name = Packages; + sourceTree = ""; + }; + DD4AF31F75D7D5BA45059FDA /* Products */ = { + isa = PBXGroup; + children = ( + 69F4D6841A944F1883A47736 /* JevDK.app */, + ); + name = Products; + sourceTree = ""; + }; +/* End PBXGroup section */ + +/* Begin PBXNativeTarget section */ + 0629784CF4F75450FD50F614 /* JevDK */ = { + isa = PBXNativeTarget; + buildConfigurationList = 9F969EB71EAA1BD410AB97A6 /* Build configuration list for PBXNativeTarget "JevDK" */; + buildPhases = ( + 909132269B768AF878F6E1CF /* Sources */, + 619937F107583FC0C4C18B60 /* Frameworks */, + ); + buildRules = ( + ); + dependencies = ( + ); + name = JevDK; + packageProductDependencies = ( + A76F927BDDBECBFD1D95BACB /* OpenJevKit */, + 4AD390EEAA615C4C5D4C5418 /* LocalLMLabSDKCore */, + 08F14639708048E2EF3B8497 /* LocalLMLabSDKRemote */, + B3CB5FCBA59A48CE738CD765 /* LocalLMLabSDKInference */, + ); + productName = JevDK; + productReference = 69F4D6841A944F1883A47736 /* JevDK.app */; + productType = "com.apple.product-type.application"; + }; +/* End PBXNativeTarget section */ + +/* Begin PBXProject section */ + 2F2626A3DCCE320918B14B92 /* Project object */ = { + isa = PBXProject; + attributes = { + BuildIndependentTargetsInParallel = YES; + LastUpgradeCheck = 1430; + TargetAttributes = { + 0629784CF4F75450FD50F614 = { + ProvisioningStyle = Manual; + }; + }; + }; + buildConfigurationList = 93B28FDF49E38E68301F8D76 /* Build configuration list for PBXProject "JevDK" */; + developmentRegion = en; + hasScannedForEncodings = 0; + knownRegions = ( + Base, + en, + ); + mainGroup = A64CD306E30FE5B113DA1DAF; + minimizedProjectReferenceProxies = 1; + packageReferences = ( + E42A7DF28484AE95E8415932 /* XCLocalSwiftPackageReference "." */, + ); + preferredProjectObjectVersion = 77; + productRefGroup = DD4AF31F75D7D5BA45059FDA /* Products */; + projectDirPath = ""; + projectRoot = ""; + targets = ( + 0629784CF4F75450FD50F614 /* JevDK */, + ); + }; +/* End PBXProject section */ + +/* Begin PBXSourcesBuildPhase section */ + 909132269B768AF878F6E1CF /* Sources */ = { + isa = PBXSourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 01C38D4ACEE528CDA8C678B0 /* AppModel.swift in Sources */, + 8BABC31DA33A9AFFC2F0DA55 /* Calibration.swift in Sources */, + 545585A403260A8CD9BB82AB /* CalibrationSheet.swift in Sources */, + 797DFE0A2A40F77BBD987A35 /* EditorView.swift in Sources */, + B3F706C436621FC2C4E031C5 /* Fonts.swift in Sources */, + 6C032E883ED0732D4FA1E543 /* HostedBackends.swift in Sources */, + F226A1630946764D74AA1773 /* JevDKApp.swift in Sources */, + 54EAB33CF8CCFD4279D2FB6F /* Keychain.swift in Sources */, + 41CDC20EF81297CAE99A72F9 /* ModelsSheet.swift in Sources */, + 18CD55EBA24EC88C2B7F802F /* Presets.swift in Sources */, + F141C2660F85B1C1CA748204 /* ResultsCSV.swift in Sources */, + DDD7A7D0558AB7E447228BDC /* ResultsView.swift in Sources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXSourcesBuildPhase section */ + +/* Begin XCBuildConfiguration section */ + 071DA1DBAA15C316F8D13FA4 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_ENABLE_OBJC_WEAK = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = dwarf; + ENABLE_STRICT_OBJC_MSGSEND = YES; + ENABLE_TESTABILITY = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_DYNAMIC_NO_PIC = NO; + GCC_NO_COMMON_BLOCKS = YES; + GCC_OPTIMIZATION_LEVEL = 0; + GCC_PREPROCESSOR_DEFINITIONS = ( + "$(inherited)", + "DEBUG=1", + ); + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + MACOSX_DEPLOYMENT_TARGET = 27.0; + MTL_ENABLE_DEBUG_INFO = INCLUDE_SOURCE; + MTL_FAST_MATH = YES; + ONLY_ACTIVE_ARCH = YES; + PRODUCT_NAME = "$(TARGET_NAME)"; + SDKROOT = macosx; + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; + SWIFT_OPTIMIZATION_LEVEL = "-Onone"; + SWIFT_VERSION = 5.0; + }; + name = Debug; + }; + 546E9494D61E5F8942F71518 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_IDENTITY = "-"; + CODE_SIGN_STYLE = Manual; + COMBINE_HIDPI_IMAGES = YES; + CURRENT_PROJECT_VERSION = 1; + ENABLE_APP_SANDBOX = NO; + ENABLE_HARDENED_RUNTIME = NO; + INFOPLIST_FILE = "xcodeproj/JevDK-Info.plist"; + LD_RUNPATH_SEARCH_PATHS = ( + "@executable_path/../Frameworks", + ); + MARKETING_VERSION = 0.1.0; + PRODUCT_BUNDLE_IDENTIFIER = lab.locallm.sdk.reference.jevdk; + SDKROOT = macosx; + SWIFT_VERSION = 5.0; + }; + name = Debug; + }; + 9A108C7F603C45D4E5F1EC86 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_CXX_LANGUAGE_STANDARD = "gnu++14"; + CLANG_CXX_LIBRARY = "libc++"; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_ENABLE_OBJC_WEAK = YES; + CLANG_WARN_BLOCK_CAPTURE_AUTORELEASING = YES; + CLANG_WARN_BOOL_CONVERSION = YES; + CLANG_WARN_COMMA = YES; + CLANG_WARN_CONSTANT_CONVERSION = YES; + CLANG_WARN_DEPRECATED_OBJC_IMPLEMENTATIONS = YES; + CLANG_WARN_DIRECT_OBJC_ISA_USAGE = YES_ERROR; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_EMPTY_BODY = YES; + CLANG_WARN_ENUM_CONVERSION = YES; + CLANG_WARN_INFINITE_RECURSION = YES; + CLANG_WARN_INT_CONVERSION = YES; + CLANG_WARN_NON_LITERAL_NULL_CONVERSION = YES; + CLANG_WARN_OBJC_IMPLICIT_RETAIN_SELF = YES; + CLANG_WARN_OBJC_LITERAL_CONVERSION = YES; + CLANG_WARN_OBJC_ROOT_CLASS = YES_ERROR; + CLANG_WARN_QUOTED_INCLUDE_IN_FRAMEWORK_HEADER = YES; + CLANG_WARN_RANGE_LOOP_ANALYSIS = YES; + CLANG_WARN_STRICT_PROTOTYPES = YES; + CLANG_WARN_SUSPICIOUS_MOVE = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + CLANG_WARN_UNREACHABLE_CODE = YES; + CLANG_WARN__DUPLICATE_METHOD_MATCH = YES; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; + ENABLE_NS_ASSERTIONS = NO; + ENABLE_STRICT_OBJC_MSGSEND = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_NO_COMMON_BLOCKS = YES; + GCC_WARN_64_TO_32_BIT_CONVERSION = YES; + GCC_WARN_ABOUT_RETURN_TYPE = YES_ERROR; + GCC_WARN_UNDECLARED_SELECTOR = YES; + GCC_WARN_UNINITIALIZED_AUTOS = YES_AGGRESSIVE; + GCC_WARN_UNUSED_FUNCTION = YES; + GCC_WARN_UNUSED_VARIABLE = YES; + MACOSX_DEPLOYMENT_TARGET = 27.0; + MTL_ENABLE_DEBUG_INFO = NO; + MTL_FAST_MATH = YES; + PRODUCT_NAME = "$(TARGET_NAME)"; + SDKROOT = macosx; + SWIFT_COMPILATION_MODE = wholemodule; + SWIFT_OPTIMIZATION_LEVEL = "-O"; + SWIFT_VERSION = 5.0; + }; + name = Release; + }; + A1C570C0E1E4F2FEE01B06EB /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + CODE_SIGN_IDENTITY = "-"; + CODE_SIGN_STYLE = Manual; + COMBINE_HIDPI_IMAGES = YES; + CURRENT_PROJECT_VERSION = 1; + ENABLE_APP_SANDBOX = NO; + ENABLE_HARDENED_RUNTIME = NO; + INFOPLIST_FILE = "xcodeproj/JevDK-Info.plist"; + LD_RUNPATH_SEARCH_PATHS = ( + "@executable_path/../Frameworks", + ); + MARKETING_VERSION = 0.1.0; + PRODUCT_BUNDLE_IDENTIFIER = lab.locallm.sdk.reference.jevdk; + SDKROOT = macosx; + SWIFT_VERSION = 5.0; + }; + name = Release; + }; +/* End XCBuildConfiguration section */ + +/* Begin XCConfigurationList section */ + 93B28FDF49E38E68301F8D76 /* Build configuration list for PBXProject "JevDK" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 071DA1DBAA15C316F8D13FA4 /* Debug */, + 9A108C7F603C45D4E5F1EC86 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Debug; + }; + 9F969EB71EAA1BD410AB97A6 /* Build configuration list for PBXNativeTarget "JevDK" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 546E9494D61E5F8942F71518 /* Debug */, + A1C570C0E1E4F2FEE01B06EB /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Debug; + }; +/* End XCConfigurationList section */ + +/* Begin XCLocalSwiftPackageReference section */ + E42A7DF28484AE95E8415932 /* XCLocalSwiftPackageReference "." */ = { + isa = XCLocalSwiftPackageReference; + relativePath = .; + }; +/* End XCLocalSwiftPackageReference section */ + +/* Begin XCSwiftPackageProductDependency section */ + 08F14639708048E2EF3B8497 /* LocalLMLabSDKRemote */ = { + isa = XCSwiftPackageProductDependency; + productName = LocalLMLabSDKRemote; + }; + 4AD390EEAA615C4C5D4C5418 /* LocalLMLabSDKCore */ = { + isa = XCSwiftPackageProductDependency; + productName = LocalLMLabSDKCore; + }; + A76F927BDDBECBFD1D95BACB /* OpenJevKit */ = { + isa = XCSwiftPackageProductDependency; + productName = OpenJevKit; + }; + B3CB5FCBA59A48CE738CD765 /* LocalLMLabSDKInference */ = { + isa = XCSwiftPackageProductDependency; + productName = LocalLMLabSDKInference; + }; +/* End XCSwiftPackageProductDependency section */ + }; + rootObject = 2F2626A3DCCE320918B14B92 /* Project object */; +} diff --git a/examples/jevdk/JevDK.xcodeproj/project.xcworkspace/contents.xcworkspacedata b/examples/jevdk/JevDK.xcodeproj/project.xcworkspace/contents.xcworkspacedata new file mode 100644 index 0000000..919434a --- /dev/null +++ b/examples/jevdk/JevDK.xcodeproj/project.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,7 @@ + + + + + diff --git a/examples/jevdk/JevDK.xcodeproj/xcshareddata/xcschemes/JevDK.xcscheme b/examples/jevdk/JevDK.xcodeproj/xcshareddata/xcschemes/JevDK.xcscheme new file mode 100644 index 0000000..c7b9d07 --- /dev/null +++ b/examples/jevdk/JevDK.xcodeproj/xcshareddata/xcschemes/JevDK.xcscheme @@ -0,0 +1,91 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/examples/jevdk/Package.swift b/examples/jevdk/Package.swift index ff27e76..8ff7554 100644 --- a/examples/jevdk/Package.swift +++ b/examples/jevdk/Package.swift @@ -64,6 +64,11 @@ let package = Package( platforms: [.macOS("27.0")], products: [ .library(name: "OpenJevKit", targets: ["OpenJevKit"]), + // Vend the binaries as library products so the XcodeGen .xcodeproj variant (project.yml) + // can depend on them by name. `swift build` doesn't need this. + .library(name: "LocalLMLabSDKCore", targets: ["LocalLMLabSDKCore"]), + .library(name: "LocalLMLabSDKRemote", targets: ["LocalLMLabSDKRemote"]), + .library(name: "LocalLMLabSDKInference", targets: ["LocalLMLabSDKInference"]), ], targets: [ .binaryTarget(name: "LocalLMLabSDKCore", url: sdk.coreURL, checksum: sdk.coreChecksum), diff --git a/examples/jevdk/project.yml b/examples/jevdk/project.yml new file mode 100644 index 0000000..5034523 --- /dev/null +++ b/examples/jevdk/project.yml @@ -0,0 +1,68 @@ +# XcodeGen spec for the "double-click, Run" variant of JevDK. +# +# brew install xcodegen && xcodegen generate # regenerates JevDK.xcodeproj +# +# The generated JevDK.xcodeproj IS committed — a fresh clone opens it and Runs. +# Edit project.yml, not the .xcodeproj, then regenerate. +# +# Not sandboxed, like `swift build`: JevDK lists and verifies models in the shared Hugging Face +# cache (~/.cache/huggingface/hub, or HF_HUB_CACHE / HF_HOME). Hosted-backend keys come from the +# Keychain (saved in Backends), or OPENROUTER_API_KEY / FEATHERLESS_API_KEY in the environment +# (Edit Scheme > Run > Environment Variables, kept out of this committed scheme). +# +# Info.plist is written by XcodeGen to xcodeproj/JevDK-Info.plist (committed) from the +# `properties` below. +name: JevDK +options: + bundleIdPrefix: lab.locallm.sdk.reference + deploymentTarget: + macOS: "27.0" + createIntermediateGroups: true + groupSortPosition: top +schemes: + JevDK: + build: + targets: + JevDK: all + run: + config: Debug +packages: + JevDKPackage: + path: . +targets: + JevDK: + type: application + platform: macOS + sources: + - path: Sources/JevDK + info: + path: xcodeproj/JevDK-Info.plist + properties: + CFBundleDisplayName: JevDK + CFBundleName: JevDK + LSApplicationCategoryType: public.app-category.developer-tools + LSMinimumSystemVersion: "27.0" + NSHighResolutionCapable: true + settings: + base: + PRODUCT_BUNDLE_IDENTIFIER: lab.locallm.sdk.reference.jevdk + MARKETING_VERSION: "0.1.0" + CURRENT_PROJECT_VERSION: "1" + SWIFT_VERSION: "5.0" + # Ad-hoc, no sandbox, no hardened runtime — this-Mac only, the same tier as model-switch. + # Hardened runtime would reject the Team-signed SDK frameworks under an ad-hoc outer signature. + CODE_SIGN_IDENTITY: "-" + CODE_SIGN_STYLE: Manual + ENABLE_APP_SANDBOX: NO + ENABLE_HARDENED_RUNTIME: NO + LD_RUNPATH_SEARCH_PATHS: + - "@executable_path/../Frameworks" + dependencies: + - package: JevDKPackage + product: OpenJevKit + - package: JevDKPackage + product: LocalLMLabSDKCore + - package: JevDKPackage + product: LocalLMLabSDKRemote + - package: JevDKPackage + product: LocalLMLabSDKInference diff --git a/examples/jevdk/xcodeproj/JevDK-Info.plist b/examples/jevdk/xcodeproj/JevDK-Info.plist new file mode 100644 index 0000000..cf1d69e --- /dev/null +++ b/examples/jevdk/xcodeproj/JevDK-Info.plist @@ -0,0 +1,30 @@ + + + + + CFBundleDevelopmentRegion + $(DEVELOPMENT_LANGUAGE) + CFBundleDisplayName + JevDK + CFBundleExecutable + $(EXECUTABLE_NAME) + CFBundleIdentifier + $(PRODUCT_BUNDLE_IDENTIFIER) + CFBundleInfoDictionaryVersion + 6.0 + CFBundleName + JevDK + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleVersion + 1 + LSApplicationCategoryType + public.app-category.developer-tools + LSMinimumSystemVersion + 27.0 + NSHighResolutionCapable + + + From 65a9fca628059a8ee09f2abac1947ae456e6678e Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 19:42:50 -0700 Subject: [PATCH 28/73] jevdk: no key file; keys from the Keychain or the environment --check-remote read API keys from ./.env. JevDK no longer reads any key file: keys come from the Keychain (saved in Backends) or, for CI, OPENROUTER_API_KEY / FEATHERLESS_API_KEY. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/README.md | 11 ++++++-- examples/jevdk/Sources/JevDK/AppModel.swift | 4 +-- .../jevdk/Sources/JevDK/HostedBackends.swift | 25 ++++++++----------- examples/jevdk/Sources/JevDK/JevDKApp.swift | 6 ++--- 4 files changed, 25 insertions(+), 21 deletions(-) diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index e5c4f38..8a65a15 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -14,6 +14,12 @@ Needs an Apple-silicon Mac, macOS 27 and Xcode 27 (Swift 6.4). `Package.swift` b No Metal Toolchain needed; the prebuilt Inference xcframework bundles the compiled shaders. To build against another release, set `LOCALLM_SDK_VERSION` (see [the examples README](../README.md)). +**Or open it in Xcode and Run:** `open -a Xcode JevDK.xcodeproj`, scheme **JevDK**. It's ad-hoc +signed ("Sign to Run Locally") and not sandboxed, like the `swift build` binary, so it shares your +Hugging Face cache and Keychain keys. Generated from [`project.yml`](project.yml) with +[XcodeGen](https://github.com/yonaskolb/XcodeGen): edit `project.yml`, not the `.xcodeproj`, then +`xcodegen generate`. + **New to decision models?** Read [the developer's guide](GUIDE.md) first: what a decider is, when to use one, and the workflow from writing questions to shipping them, with a worked example. @@ -174,6 +180,7 @@ Headless checks: ``` `--check` needs a model the SDK has downloaded (or verified). `--check-remote` takes -`featherlessDemo`, `featherless` or `openRouter`, with keys from the environment or a -git-ignored `./.env` (`OPENROUTER_API_KEY`, `FEATHERLESS_API_KEY`), never printed. The window +`featherlessDemo`, `featherless` or `openRouter`, using the key saved in **Backends** (the +Keychain) or, for CI, the `OPENROUTER_API_KEY` / `FEATHERLESS_API_KEY` environment variable. Keys +are never printed, and never read from a file. The window hasn't been driven by a test. diff --git a/examples/jevdk/Sources/JevDK/AppModel.swift b/examples/jevdk/Sources/JevDK/AppModel.swift index 6ebdab4..157ba9f 100644 --- a/examples/jevdk/Sources/JevDK/AppModel.swift +++ b/examples/jevdk/Sources/JevDK/AppModel.swift @@ -143,8 +143,8 @@ final class AppModel { } } - func apiKey(_ b: HostedBackend) -> String? { Keychain.get("key.\(b.rawValue)") } - func setAPIKey(_ key: String?, _ b: HostedBackend) { Keychain.set(key, for: "key.\(b.rawValue)") } + func apiKey(_ b: HostedBackend) -> String? { Keychain.get(APIKeys.account(b)) } + func setAPIKey(_ key: String?, _ b: HostedBackend) { Keychain.set(key, for: APIKeys.account(b)) } var enabledHosted: [HostedBackend] { HostedBackend.allCases.filter { hostedEnabled[$0] == true } } diff --git a/examples/jevdk/Sources/JevDK/HostedBackends.swift b/examples/jevdk/Sources/JevDK/HostedBackends.swift index 40e5830..84631d1 100644 --- a/examples/jevdk/Sources/JevDK/HostedBackends.swift +++ b/examples/jevdk/Sources/JevDK/HostedBackends.swift @@ -27,7 +27,7 @@ enum HostedBackend: String, CaseIterable, Identifiable, Sendable { var needsKey: Bool { self != .featherlessDemo } - /// Environment / .env variable for headless runs. + /// Environment variable checked when no key is saved in the Keychain (CI, headless runs). var keyVariable: String? { switch self { case .featherlessDemo: nil @@ -82,18 +82,15 @@ enum HostedBackend: String, CaseIterable, Identifiable, Sendable { } } -/// Reads an API key from the environment or a `.env` file, never printing it. Accepts -/// `NAME=value` lines or a file holding just the key. -enum KeyFile { - static func key(named name: String, file: URL = URL(fileURLWithPath: ".env")) -> String? { - if let v = ProcessInfo.processInfo.environment[name], !v.isEmpty { return v } - guard let text = try? String(contentsOf: file, encoding: .utf8) else { return nil } - let lines = text.split(whereSeparator: \.isNewline).map { $0.trimmingCharacters(in: .whitespaces) } - for line in lines where line.hasPrefix(name + "=") || line.hasPrefix("export " + name + "=") { - let v = line.split(separator: "=", maxSplits: 1).last.map(String.init) ?? "" - return v.trimmingCharacters(in: CharacterSet(charactersIn: "\"' ")) - } - let bare = lines.filter { !$0.isEmpty && !$0.hasPrefix("#") } - return bare.count == 1 && !bare[0].contains("=") ? bare[0] : nil +/// The API key for a hosted backend: the one saved in the Keychain from **Backends**, else the +/// environment variable (for CI and headless runs). Never printed. There is no key file: a key +/// in a file next to the code is too easy to commit or share. +enum APIKeys { + static func key(for backend: HostedBackend) -> String? { + if let k = Keychain.get(account(backend)), !k.isEmpty { return k } + guard let name = backend.keyVariable, let v = ProcessInfo.processInfo.environment[name], !v.isEmpty else { return nil } + return v } + + static func account(_ backend: HostedBackend) -> String { "key.\(backend.rawValue)" } } diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index 09655c6..a87cbf6 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -18,8 +18,8 @@ struct JevDKApp: App { HeadlessCheck.run(preset: args[i + 1], modelDir: args[i + 2]) } // `JevDK --check-remote [model]`: the - // same, against a hosted Jev. Keys come from the environment or ./.env - // (FEATHERLESS_API_KEY / OPENROUTER_API_KEY) and are never printed. + // same, against a hosted Jev. Keys come from the Keychain (saved in Backends) or the + // environment (FEATHERLESS_API_KEY / OPENROUTER_API_KEY) and are never printed. if let i = args.firstIndex(of: "--check-remote"), args.count > i + 2 { HeadlessCheck.runRemote(preset: args[i + 1], provider: args[i + 2], model: args.count > i + 3 && !args[i + 3].hasPrefix("--") ? args[i + 3] : nil) @@ -186,7 +186,7 @@ enum HeadlessCheck { guard let p = preset(name), let backend = HostedBackend(rawValue: provider) else { print("unknown preset or provider (have: \(HostedBackend.allCases.map(\.rawValue)))"); exit(1) } - let key = backend.keyVariable.flatMap { KeyFile.key(named: $0) } + let key = APIKeys.key(for: backend) if backend.needsKey { print("key: \(key == nil ? "MISSING" : "found (\(key!.count) chars)")") } let lab = LocalLMLab() try? lab.models.register(decision: JevDecisionProvider(backend.config(key: key))) From 290c7589521024fcfcc36eaa83085d1872eee14e Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 3 Oct 2026 19:53:23 -0700 Subject: [PATCH 29/73] examples: jevdk release script for a signed, notarized DMG scripts/release_macos.sh archives JevDK.xcodeproj (Release, arm64, unsigned), re-signs the embedded SDK frameworks inside-out and the app with a Developer ID identity and the hardened runtime, notarizes and staples the app and a drag-to-Applications DMG, and writes its sha256. Same options as locallmlab/scripts/release-macos.sh. README points to it. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/README.md | 3 + examples/jevdk/scripts/release_macos.sh | 240 ++++++++++++++++++++++++ 2 files changed, 243 insertions(+) create mode 100755 examples/jevdk/scripts/release_macos.sh diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index 8a65a15..d2aa687 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -20,6 +20,9 @@ Hugging Face cache and Keychain keys. Generated from [`project.yml`](project.yml [XcodeGen](https://github.com/yonaskolb/XcodeGen): edit `project.yml`, not the `.xcodeproj`, then `xcodegen generate`. +**A signed, notarized DMG to share:** `VERSION=x.y.z APP_IDENTITY=… KEYCHAIN_PROFILE=… scripts/release_macos.sh` +(Developer ID identity, `notarytool` profile; options in the script's header) → `dist/JevDK-x.y.z-arm64.dmg`. + **New to decision models?** Read [the developer's guide](GUIDE.md) first: what a decider is, when to use one, and the workflow from writing questions to shipping them, with a worked example. diff --git a/examples/jevdk/scripts/release_macos.sh b/examples/jevdk/scripts/release_macos.sh new file mode 100755 index 0000000..9f568d0 --- /dev/null +++ b/examples/jevdk/scripts/release_macos.sh @@ -0,0 +1,240 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Builds a Developer ID–signed, notarized, stapled JevDK.app and a drag-to-Applications DMG. +# +# VERSION=0.1.0 \ +# APP_IDENTITY="Developer ID Application: Your Name (TEAMID)" \ +# KEYCHAIN_PROFILE=notary-profile \ +# scripts/release_macos.sh +# +# VERSION required, x.y.z — written to CFBundleShortVersionString / CFBundleVersion +# APP_IDENTITY required, a "Developer ID Application" identity (name or SHA-1 hash; use the +# hash when several identities share a name). SIGN_IDENTITY also accepted. +# KEYCHAIN_PROFILE notarytool profile (xcrun notarytool store-credentials ); required +# unless both NOTARIZE_* are 0. NOTARY_PROFILE also accepted. +# TEAM_ID optional, passed to notarytool +# NOTARIZE_APP 1 (default) | 0 +# NOTARIZE_DMG 1 (default) | 0 +# DEVELOPER_DIR Xcode 27 (default: xcode-select -p) +# +# Builds through JevDK.xcodeproj (project.yml) with signing off, then signs everything itself, +# inside-out, with the hardened runtime: the embedded SDK frameworks (Core, Remote, Inference) +# and their nested resource bundles, then the app. Not sandboxed, no entitlements — same as the +# `swift build` binary and the Xcode Run (JevDK uses the shared Hugging Face cache). +# +# Artifacts: dist/JevDK.app, dist/JevDK--arm64.dmg (+ .sha256). Scratch in build/release. + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +APP_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" + +APP_NAME="JevDK" +SCHEME="JevDK" +VERSION="${VERSION:-VERSION_NEEDED}" +APP_IDENTITY="${APP_IDENTITY:-${SIGN_IDENTITY:-}}" +KEYCHAIN_PROFILE="${KEYCHAIN_PROFILE:-${NOTARY_PROFILE:-}}" +DEVELOPER_DIR="${DEVELOPER_DIR:-$(xcode-select -p)}" +TEAM_ID="${TEAM_ID:-}" +NOTARIZE_APP="${NOTARIZE_APP:-1}" +NOTARIZE_DMG="${NOTARIZE_DMG:-1}" + +DIST_DIR="$APP_ROOT/dist" +BUILD_DIR="$APP_ROOT/build/release" +ARCHIVE_PATH="$BUILD_DIR/${APP_NAME}.xcarchive" +APP_DIR="$DIST_DIR/${APP_NAME}.app" +CONTENTS_DIR="$APP_DIR/Contents" +FRAMEWORKS_DIR="$CONTENTS_DIR/Frameworks" +DMG_STAGE_DIR="$BUILD_DIR/dmg-stage" +APP_ZIP="$BUILD_DIR/${APP_NAME}-${VERSION}.zip" +DMG_PATH="$DIST_DIR/${APP_NAME}-${VERSION}-arm64.dmg" + +export DEVELOPER_DIR + +require_env() { + local name="$1" + local value="$2" + if [[ -z "$value" ]]; then + echo "$name is required" >&2 + exit 1 + fi +} + +require_command() { + local command_name="$1" + if ! command -v "$command_name" >/dev/null 2>&1; then + echo "$command_name is required" >&2 + exit 1 + fi +} + +notarize_and_wait() { + local artifact="$1" + echo "Submitting for notarization: $artifact" + if ! xcrun notarytool history --keychain-profile "$KEYCHAIN_PROFILE" >/dev/null 2>&1; then + echo "KEYCHAIN_PROFILE became unavailable before notarizing: $artifact" >&2 + echo "Try unlocking the login keychain or recreating the profile with xcrun notarytool store-credentials." >&2 + exit 1 + fi + if [[ -n "$TEAM_ID" ]]; then + xcrun notarytool submit "$artifact" \ + --keychain-profile "$KEYCHAIN_PROFILE" \ + --team-id "$TEAM_ID" \ + --wait + else + xcrun notarytool submit "$artifact" \ + --keychain-profile "$KEYCHAIN_PROFILE" \ + --wait + fi +} + +sign() { + codesign --force --options runtime --timestamp --sign "$APP_IDENTITY" "$@" +} + +require_env VERSION "$VERSION" +if [[ "$VERSION" == "VERSION_NEEDED" ]]; then + echo "Set VERSION=x.y.z" >&2 + exit 1 +fi +require_env APP_IDENTITY "$APP_IDENTITY" +if [[ "$NOTARIZE_APP" == "1" || "$NOTARIZE_DMG" == "1" ]]; then + require_env KEYCHAIN_PROFILE "$KEYCHAIN_PROFILE" +fi + +require_command xcodebuild +require_command xattr +require_command codesign +require_command ditto +require_command hdiutil +require_command shasum +require_command spctl +require_command xcrun + +if ! security find-identity -v -p codesigning | grep -F "$APP_IDENTITY" >/dev/null 2>&1; then + echo "APP_IDENTITY is not installed or is not valid for codesigning: $APP_IDENTITY" >&2 + security find-identity -v -p codesigning || true + exit 1 +fi +if [[ "$(security find-identity -v -p codesigning | grep -cF "$APP_IDENTITY")" -gt 1 ]]; then + echo "APP_IDENTITY matches more than one identity; pass its SHA-1 hash instead: $APP_IDENTITY" >&2 + security find-identity -v -p codesigning | grep -F "$APP_IDENTITY" >&2 || true + exit 1 +fi + +if [[ "$NOTARIZE_APP" == "1" || "$NOTARIZE_DMG" == "1" ]] && ! xcrun notarytool history --keychain-profile "$KEYCHAIN_PROFILE" >/dev/null 2>&1; then + echo "KEYCHAIN_PROFILE is not usable: $KEYCHAIN_PROFILE" >&2 + echo "Create it with: xcrun notarytool store-credentials $KEYCHAIN_PROFILE" >&2 + exit 1 +fi + +echo "Cleaning release artifacts..." +rm -rf "$BUILD_DIR" "$APP_DIR" "$DMG_PATH" "$DMG_PATH.sha256" +mkdir -p "$BUILD_DIR" "$DIST_DIR" + +# Release config, arm64 only, unsigned: the frameworks keep the SDK team's signature until we +# re-sign them below with APP_IDENTITY (required — the hardened runtime's library validation +# only loads frameworks signed by the app's own team). +echo "Archiving $SCHEME (Release, arm64)..." +xcodebuild archive \ + -project "$APP_ROOT/JevDK.xcodeproj" \ + -scheme "$SCHEME" \ + -configuration Release \ + -destination "generic/platform=macOS" \ + -archivePath "$ARCHIVE_PATH" \ + -derivedDataPath "$BUILD_DIR/DerivedData" \ + ARCHS=arm64 \ + ONLY_ACTIVE_ARCH=NO \ + MARKETING_VERSION="$VERSION" \ + CURRENT_PROJECT_VERSION="$VERSION" \ + CODE_SIGNING_ALLOWED=NO \ + CODE_SIGNING_REQUIRED=NO \ + CODE_SIGN_IDENTITY="" + +ditto "$ARCHIVE_PATH/Products/Applications/${APP_NAME}.app" "$APP_DIR" + +# project.yml's Info.plist carries fixed version strings; stamp the release version. +/usr/libexec/PlistBuddy -c "Set :CFBundleShortVersionString $VERSION" "$CONTENTS_DIR/Info.plist" +/usr/libexec/PlistBuddy -c "Set :CFBundleVersion $VERSION" "$CONTENTS_DIR/Info.plist" + +# Apache-2.0 §4(d): carry the SDK's NOTICE (it bundles the MLX stack) inside the app. +if [[ -f "$APP_ROOT/../../NOTICE" ]]; then + mkdir -p "$CONTENTS_DIR/Resources" + cp "$APP_ROOT/../../NOTICE" "$CONTENTS_DIR/Resources/NOTICE" +fi + +# The published xcframework zips carry AppleDouble/xattr detritus that breaks signing. +find "$APP_DIR" -name '._*' -delete +xattr -cr "$APP_DIR" + +echo "Verifying the app binary is arm64..." +file "$CONTENTS_DIR/MacOS/$APP_NAME" +shopt -s nullglob +FRAMEWORKS=("$FRAMEWORKS_DIR"/*.framework) +shopt -u nullglob +for fw in ${FRAMEWORKS[@]+"${FRAMEWORKS[@]}"}; do + file "$fw/$(basename "$fw" .framework)" +done + +echo "Signing app bundle..." +# Inside-out: each framework's nested bundles, its binary, the framework; then the app last. +for fw in ${FRAMEWORKS[@]+"${FRAMEWORKS[@]}"}; do # (empty-array safe under set -u in bash 3.2) + while IFS= read -r b; do sign "$b"; done < <(find "$fw" -depth -type d -name '*.bundle') + sign "$fw/$(basename "$fw" .framework)" + sign "$fw" +done +sign "$APP_DIR" + +codesign --verify --deep --strict --verbose=2 "$APP_DIR" + +echo "Creating app notarization zip..." +ditto -c -k --keepParent "$APP_DIR" "$APP_ZIP" + +if [[ "$NOTARIZE_APP" == "1" ]]; then + notarize_and_wait "$APP_ZIP" + + echo "Stapling app..." + xcrun stapler staple "$APP_DIR" + xcrun stapler validate "$APP_DIR" + spctl -a -vv --type execute "$APP_DIR" +else + echo "Skipping app notarization because NOTARIZE_APP=$NOTARIZE_APP" +fi + +echo "Creating drag-to-Applications DMG..." +rm -rf "$DMG_STAGE_DIR" +mkdir -p "$DMG_STAGE_DIR" +cp -R "$APP_DIR" "$DMG_STAGE_DIR/${APP_NAME}.app" +ln -s /Applications "$DMG_STAGE_DIR/Applications" + +hdiutil create \ + -volname "$APP_NAME" \ + -srcfolder "$DMG_STAGE_DIR" \ + -ov \ + -format UDZO \ + "$DMG_PATH" + +echo "Signing DMG..." +codesign --force --timestamp --sign "$APP_IDENTITY" "$DMG_PATH" +codesign --verify --verbose=2 "$DMG_PATH" + +if [[ "$NOTARIZE_DMG" == "1" ]]; then + notarize_and_wait "$DMG_PATH" + + echo "Stapling DMG..." + xcrun stapler staple "$DMG_PATH" + xcrun stapler validate "$DMG_PATH" + + echo "Running Gatekeeper verification..." + spctl --assess --type open --context context:primary-signature --verbose "$DMG_PATH" +else + echo "Skipping DMG notarization because NOTARIZE_DMG=$NOTARIZE_DMG" +fi + +# Write the checksum with a bare filename (not the absolute build path) so a +# consumer can `shasum -a 256 -c .sha256` from the directory it's in. +( cd "$DIST_DIR" && shasum -a 256 "$(basename "$DMG_PATH")" > "$(basename "$DMG_PATH").sha256" ) + +echo "Release artifacts:" +echo "$APP_DIR" +echo "$DMG_PATH" +echo "$DMG_PATH.sha256" From 6b4968de3715ae7de6c7de6f068c520d9afd4630 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 07:03:35 -0700 Subject: [PATCH 30/73] docs: decision models (Jev) in the SDK guide, migration guide, annotated examples, API surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - sdk-guide.md §6c: lab.decide, local OpenJev and hosted Jev, fidelity and calibration, one model or two (pair(decision:generator:)), question sets / answer sets / lab.evaluate, writing questions; points to the jev.html introduction. §12 reference; jevdk in Start here. - migrating-to-2.0.md: the decision API under what's new. - annotated-examples.md: jevdk, SDK-facing excerpts. - api-surface.md regenerated from the republished v2.0.0-dev binaries (decision API, maxModelCallsPerTurn). Co-Authored-By: Claude Opus 5.5 --- docs/annotated-examples.md | 229 +++++++++++++++++++++- docs/api-surface.md | 384 ++++++++++++++++++++++++++++++++++++- docs/migrating-to-2.0.md | 12 ++ docs/sdk-guide.md | 320 +++++++++++++++++++++++++++++++ 4 files changed, 938 insertions(+), 7 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index cb3ec88..152d08b 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -2,8 +2,8 @@ The full source of every reference app, with every line that actually touches the SDK marked `// ← SDK` (Core), `// ← SDK (Inference)` (the MLX runtime — `code-buddy`, `repo-qa-local`, -`workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, `mlx-control-room`, and `mcp-chat`), `// ← SDK (Remote)` (online providers — -`model-switch` and `security-demo`), `// ← SDK (MCPAppsHost)` (showing MCP servers' interactive views — +`workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, `mlx-control-room`, `mcp-chat` and `jevdk`), `// ← SDK (Remote)` (online providers — +`model-switch`, `security-demo` and `jevdk`), `// ← SDK (MCPAppsHost)` (showing MCP servers' interactive views — `mcp-chat`), or `// ← Components` (`components-demo`, `components-updates-demo`, and `mcp-chat`). Everything else is ordinary SwiftUI/Foundation — the point of marking it this way is to make obvious just how little of each file is SDK-specific plumbing. `plate-today` and `plate-today-tools` are a matched pair — the same app twice, "Path B" (hand- @@ -41,6 +41,7 @@ non-comment lines; these examples are commented far more heavily than production | [`aiql`](#examplesaiql) | 402 | 459 | a plain-English request → one read-only SQL `SELECT` over an MCP dataset → the CSV you asked for, sandboxed SwiftUI, zero fabricated values; a pinned default model and a trust policy for the free-text picker | | [`vistanova`](#examplesvistanova) | 931 | 1,294 | a tiny local search engine: web search through a Tavily MCP server on one local model, summaries from a **pinned** MLX model on another; defends against a model that skips the tool call (7 files) | | [`mcp-chat`](#examplesmcp-chat) | 938 | 1,026 | a chat with a local model where an MCP server's own interactive view (Todoist's task list) shows up under the tool call: the conversation is `hostTranscript`, the view is `MCPAppsHost` (4 files, views excerpted) | +| [`jevdk`](#examplesjevdk) | 2,529 | 2,748 | a playground for decision-model questions: local OpenJev and hosted Jev side by side through one API, marked answers, calibration, and the tested questions exported for an app (14 files, excerpted) | | [`components-updates-demo`](#examplescomponents-updates-demo) | 151 | 170 | the `Components` model **onboarding**, **update** and **versions** views, driven by simulated sources so every state is reachable | | [`mlx-control-room`](#examplesmlx-control-room) | 1,434 | 1,812 | every MLX knob with a gauge, plus the supply-chain flow made visible: validate, download, **pin**, update, roll back, clean up (excerpted; UI omitted) | @@ -3780,6 +3781,230 @@ minimal end (one `pinnedRevisions:` and nothing else). The one thing worth copyi `AllowMLXCommunityOrShipped`: an allow-list keyed on `mlx-community/` plus the exact artifacts the app itself ships, because a curated adapter lives in a different namespace. +## `examples/jevdk` + +*2,529 lines of code across 14 files (2,748 with comments). Excerpted: the SDK-facing parts of +`AppModel.swift` and `HostedBackends.swift` are below; the editor, batch grid, calibration +sheet, model browser and results-CSV writer are plain SwiftUI and Foundation and are omitted +(`// …`).* + +A playground for **decision models** ([`sdk-guide.md` §6c](sdk-guide.md#6c-decision-models-jev--labdecide); +for what a decision model is, see [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html)). +Write yes / no, choice and score questions, run them on a local MLX model and on hosted Jev side +by side, mark the right answers, calibrate, and export the tested questions for your app. Three +SDK pieces carry it: + +- **`lab.decide`** for every hosted run: each enabled backend is a `JevDecisionProvider` + registered under its own scheme with its own decision route, so the demo and the keyed + Featherless endpoint coexist in one lab. +- **`OpenJevDecisionProvider`** for local runs, built with the editor's wrapper so the system + instructions stay editable, and called through `decideWithDiagnostics` for the exact prompts + and how much probability fell on the allowed labels. It validates against the provider's limits + first, as `lab.decide` would. +- **`DecisionCalibration`, `DecisionQuestionSet`, `DecisionAnswerSet`** for the workflow around + the runs: fitting, exporting for an app, and importing / exporting marked answers as CSV. + +Links `LocalLMLabSDKCore`, `LocalLMLabSDKInference` and `LocalLMLabSDKRemote`. Not sandboxed; +API keys live in the Keychain (or, for headless runs, environment variables), never in a file. + +### `Sources/JevDK/AppModel.swift` (excerpt) + +Runs one input on every enabled backend concurrently: hosted through `lab.decide`, local through +the OpenJev provider directly. Note there is no prompt building and no answer parsing anywhere: +questions go in typed, answers come back typed. + +```swift +import AppKit +import Foundation +import LocalLMLabSDKCore // ← SDK +import LocalLMLabSDKInference // ← SDK (Inference) +import LocalLMLabSDKRemote // ← SDK (Remote) +import Observation +import OpenJevKit +import UniformTypeIdentifiers + // … + let lab = LocalLMLab() // ← SDK + /// Local models: listing, preflight, download, and the weights OpenJev scores on. + let mlx = MLXModelProvider() // ← SDK (Inference) + private var batchTask: Task? + // … + func refreshModels() { + let probe = OpenJevDecisionProvider(mlx: mlx) // ← SDK (Inference) + models = mlx.installed.map { m in // ← SDK (Inference) + LocalChoice(repoID: m.repoID, sizeBytes: m.sizeBytes, + isMoE: probe.warnings(for: Self.openJevID(m.repoID)).contains(.mixtureOfExperts)) // ← SDK (Inference) + }.sorted { ($0.sizeBytes ?? 0) < ($1.sizeBytes ?? 0) } + // … + private struct HostedPlan { + let route: RouteName // ← SDK + let label: String + let requestPreview: String + let info: ResultsCSV.BackendInfo + } + + /// Register the enabled hosted backends and point a decision route at each. Re-registered every + /// run so a changed key or model takes effect. + private func hostedPlans(for request: DecisionRequest) -> [HostedPlan] { // ← SDK + enabledHosted.compactMap { b in + let model = hostedModel[b] ?? b.defaultModel + guard let id = b.modelID(model) else { return nil } + lab.models.replace(decision: JevDecisionProvider(b.config(key: b.needsKey ? apiKey(b) : nil))) // ← SDK (Remote) + let route = RouteName("hosted.\(b.rawValue)") // ← SDK + lab.models.route(decision: route, to: id) // ← SDK + let body = (try? JevDecisionProvider.requestBody(model: model, request: request)) // ← SDK (Remote) + .flatMap { try? JSONSerialization.jsonObject(with: $0) } + .flatMap { try? JSONSerialization.data(withJSONObject: $0, options: [.prettyPrinted]) } + .map { String(decoding: $0, as: UTF8.self) } ?? "" + return HostedPlan(route: route, label: hostedLabel(b), + requestPreview: "POST \(b.config(key: nil).endpoint.absoluteString)\n\(body)", + info: .init(backend: b.rawValue, model: model, revision: nil, sizeBytes: nil, isMoE: nil, + wrapperLabel: nil, wrapperSystem: nil)) + } + } + + /// Run `input` on every enabled backend, concurrently. + private func runAll(_ input: String) async -> [BackendRun] { + let questions = set.questions + let request = DecisionRequest(state: .text(input), questions: questions.map(\.sdkQuestion)) // ← SDK + let plans = hostedPlans(for: request) + let local: (LocalChoice, OpenJevDecisionProvider)? = useLocal ? selectedModel.map { m in // ← SDK (Inference) + (m, OpenJevDecisionProvider(mlx: mlx, wrapper: OpenJevWrapper(system: set.system, inputLabel: set.inputLabel))) // ← SDK (Inference) + } : nil + let localLabel = self.localLabel + let localInfo = local.map { m, openjev in + ResultsCSV.BackendInfo(backend: "local", model: m.repoID, + revision: mlx.installed.first { $0.repoID == m.repoID }?.resolvedRevision, // ← SDK (Inference) + sizeBytes: m.sizeBytes, isMoE: m.isMoE, + wrapperLabel: openjev.wrapper.inputLabel, wrapperSystem: openjev.wrapper.system) + } + + return await withTaskGroup(of: (Int, BackendRun).self) { group in + if let (m, openjev) = local { + // Local runs call the provider directly for its diagnostics (exact prompts, label + // mass), validating against its limits first as lab.decide would. + group.addTask { + do { + try request.validate(against: openjev.limits) // ← SDK + let (d, diag) = try await openjev.decideWithDiagnostics(request, using: Self.openJevID(m.repoID)) // ← SDK (Inference) + let qd = diag.questions.mapValues { + QuestionDiagnostics(labels: $0.labels, prompt: $0.prompt, labelMass: $0.labelMass, + milliseconds: Self.ms($0.latency)) + } + let rows = QuestionResult.rows(for: questions, decision: d, diagnostics: qd) + return (0, BackendRun(label: localLabel, result: DecisionResult( + input: input, results: rows, sharedPrefixTokens: diag.sharedPrefixTokens, + milliseconds: Self.ms(d.latency), backend: localLabel), info: localInfo)) + } catch { + return (0, BackendRun(label: localLabel, error: error.localizedDescription, info: localInfo)) + } + } + } + for (i, plan) in plans.enumerated() { + group.addTask { @MainActor in + do { + let d = try await self.lab.decide(route: plan.route, state: .text(input), questions: request.questions) // ← SDK + let rows = QuestionResult.rows(for: questions, decision: d, prompt: plan.requestPreview) + return (i + 1, BackendRun(label: plan.label, result: DecisionResult( + input: input, results: rows, sharedPrefixTokens: 0, milliseconds: Self.ms(d.latency), + backend: plan.label, usage: d.usage?.summary), info: plan.info, + inputTokens: d.usage?.inputTokens, costUSD: d.usage?.cost)) + } catch { + return (i + 1, BackendRun(label: plan.label, error: error.localizedDescription, info: plan.info)) + } + } + } + var out: [(Int, BackendRun)] = [] + for await r in group { out.append(r) } + return out.sorted { $0.0 < $1.0 }.map(\.1) + } + // … + /// Fit on the marked answers with the SDK's `DecisionCalibration.fit`. + func fitCalibration() { + guard let m = selectedModel else { errorMessage = "Pick a local model first."; return } + let samples = calibrationSamples + guard !samples.isEmpty else { errorMessage = "Mark some correct answers in the batch grid first."; return } + let fitted = DecisionCalibration.fit(samples) // ← SDK + calibration = FittedCalibration( + noulTemperature: fitted.noulTemperature, choiceTemperature: fitted.choiceTemperature, + scoreTemperature: fitted.scoreTemperature, repoID: m.repoID, revision: selectedRevision ?? "?", + system: set.system, inputLabel: set.inputLabel, fittedAt: Date(), + samples: Dictionary(grouping: samples, by: \.kind.rawValue).mapValues(\.count)) + applyCalibration = true + } + // … + // MARK: Question set for the app, answer sets + + /// What the questions are being tested with, for the exported question set: the local model, + /// its revision, the wrapper and (when it matches) the calibration. Nil with Local off. + var exportTuning: DecisionQuestionSet.Tuning? { // ← SDK + guard useLocal, let m = selectedModel else { return nil } + let cal = calibrationMismatch == nil ? calibration?.sdk : nil + return .init(model: Self.openJevID(m.repoID), revision: selectedRevision, system: set.system, + inputLabel: set.inputLabel, calibration: cal, testedAt: Date()) + } + + /// The questions as the SDK's `DecisionQuestionSet` JSON, for the app to bundle and load, so + /// the shipped questions are exactly the tested ones. + func exportForApp() { + if let q = set.questions.first(where: { !$0.problems.isEmpty }) { + errorMessage = "Fix “\(q.name)” first: \(q.problems.joined(separator: "; "))." + return + } + let panel = NSSavePanel() + panel.allowedContentTypes = [.json] + let base = set.name.isEmpty ? "questions" : set.name.lowercased().replacingOccurrences(of: " ", with: "-") + panel.nameFieldStringValue = "\(base).decisions.json" + panel.message = "A question set for your app: bundle it and load it with DecisionQuestionSet(contentsOf:)." + guard panel.runModal() == .OK, let url = panel.url else { return } + do { + let file = DecisionQuestionSet(name: set.name, questions: set.questions.map(\.sdkQuestion), tuning: exportTuning) // ← SDK + try file.jsonData().write(to: url) // ← SDK + errorMessage = nil + } catch { + errorMessage = "Couldn't export: \(error.localizedDescription)" + } + } + +``` + +### `Sources/JevDK/HostedBackends.swift` (excerpt) + +The SDK presets, each given its own scheme, and where a key comes from. + +```swift + /// The SDK config, with a scheme of its own so the demo and the keyed Featherless endpoint + /// can both be registered in one lab. + func config(key: String?) -> JevProviderConfig { // ← SDK (Remote) + var c: JevProviderConfig // ← SDK (Remote) + switch self { + case .featherlessDemo: c = .featherlessDemo() // ← SDK (Remote) + case .featherless: c = .featherless(key: key ?? "") // ← SDK (Remote) + case .openRouter: c = .openRouter(key: key ?? "") // ← SDK (Remote) + } + c.scheme = rawValue.lowercased() + return c + } + // … +/// The API key for a hosted backend: the one saved in the Keychain from **Backends**, else the +/// environment variable (for CI and headless runs). Never printed. There is no key file: a key +/// in a file next to the code is too easy to commit or share. +enum APIKeys { + static func key(for backend: HostedBackend) -> String? { + if let k = Keychain.get(account(backend)), !k.isEmpty { return k } + guard let name = backend.keyVariable, let v = ProcessInfo.processInfo.environment[name], !v.isEmpty else { return nil } + return v + } + + static func account(_ backend: HostedBackend) -> String { "key.\(backend.rawValue)" } +} + +``` + +**Tally:** 13 lines Core (`// ← SDK`), 9 Inference, 8 Remote, in the excerpts +above. The decision API itself is a handful of calls (`register`/`replace(decision:)`, +`route(decision:)`, `decide`, `decideWithDiagnostics`, `DecisionCalibration.fit`, +`DecisionQuestionSet`, `DecisionAnswerSet`); the rest of the app is the playground around them. + ## `examples/mcp-chat` *938 lines of code across 4 files (1,026 with comments). `ChatModel.swift`, `WidgetSlot.swift` diff --git a/docs/api-surface.md b/docs/api-surface.md index 6b3df8a..0a8c430 100644 --- a/docs/api-surface.md +++ b/docs/api-surface.md @@ -2,7 +2,7 @@ > Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s. > Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check. -> SDK version: `"2.0.0-dev"` (the published `v2.0.0-dev` xcframeworks) · generated 2026-10-03T01:54Z +> SDK version: `"2.0.0-dev"` (the published `v2.0.0-dev` xcframeworks, republished 2026-10-04) · generated 2026-10-04T14:02Z ## LocalLMLabSDKCore @@ -579,6 +579,246 @@ nonisolated public init(_ content: GeneratedContent) throws nonisolated public init(_ content: GeneratedContent) throws nonisolated public init(_ content: GeneratedContent) throws nonisolated public init(_ content: GeneratedContent) throws +public struct DecisionCalibration : Sendable, Hashable, Codable +public var noulTemperature: Double +public var choiceTemperature: Double +public var scoreTemperature: Double +public init(noulTemperature: Double, choiceTemperature: Double, scoreTemperature: Double) +public static let none: DecisionCalibration +public struct Sample : Sendable, Hashable, Codable +public var kind: DecisionCalibration.Sample.Kind +public var probabilities: [Double] +public var correct: [Int] +public enum Kind : String, Sendable, Hashable, Codable, CaseIterable +case noul, choice, score +public init(_ kind: DecisionQuestion.Kind) +public init?(rawValue: String) +public typealias AllCases = [DecisionCalibration.Sample.Kind] +nonisolated public static var allCases: [DecisionCalibration.Sample.Kind] +public var rawValue: String +public init(kind: DecisionCalibration.Sample.Kind, probabilities: [Double], correct: [Int]) +public static func == (a: DecisionCalibration.Sample, b: DecisionCalibration.Sample) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct Measurement : Sendable, Hashable +public var count: Int +public var accuracy: Double +public var expectedCalibrationError: Double +public var logLoss: Double +public static func == (a: DecisionCalibration.Measurement, b: DecisionCalibration.Measurement) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public func temperature(for kind: DecisionCalibration.Sample.Kind) -> Double +public func apply(_ probabilities: [Double], kind: DecisionCalibration.Sample.Kind) -> [Double] +public static func apply(_ p: [Double], temperature t: Double) -> [Double] +public static func fit(_ samples: [DecisionCalibration.Sample]) -> DecisionCalibration +public func measure(_ samples: [DecisionCalibration.Sample]) -> DecisionCalibration.Measurement +public static func == (a: DecisionCalibration, b: DecisionCalibration) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct DecisionPairing : Sendable, Equatable +public var decider: ModelID +public var generator: ModelID +public var deciderBytes: Int64? +public var generatorBytes: Int64? +public var budgetBytes: Int64? +public var residentModels: Int +public var note: String? +public init(decider: ModelID, generator: ModelID, deciderBytes: Int64? = nil, generatorBytes: Int64? = nil, budgetBytes: Int64? = nil, residentModels: Int, note: String? = nil) +public static func == (a: DecisionPairing, b: DecisionPairing) -> Bool +public protocol PairableDecisionProvider : DecisionProvider +@MainActor func pair(decider: ModelID, generator: ModelID, registry: ModelRegistry) async throws -> DecisionPairing +@MainActor final public func pair(decision: RouteName, generator: RouteName) async throws -> DecisionPairing +public protocol DecisionProvider : Sendable +public func owns(_ id: ModelID) -> Bool +public var limits: DecisionLimits +public func prewarm(_ id: ModelID) async +public struct DecisionQuestionSet : Sendable, Hashable, Codable +public static let currentFormat: Int +public var format: Int +public var name: String +public var questions: [DecisionQuestion] +public var tuning: DecisionQuestionSet.Tuning? +public struct Tuning : Sendable, Hashable, Codable +public var model: ModelID? +public var revision: String? +public var system: String? +public var inputLabel: String? +public var calibration: DecisionCalibration? +public var testedAt: Date? +public init(model: ModelID? = nil, revision: String? = nil, system: String? = nil, inputLabel: String? = nil, calibration: DecisionCalibration? = nil, testedAt: Date? = nil) +public static func == (a: DecisionQuestionSet.Tuning, b: DecisionQuestionSet.Tuning) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public init(name: String, questions: [DecisionQuestion], tuning: DecisionQuestionSet.Tuning? = nil) +public init(contentsOf url: URL) throws +public init(json data: Data) throws +public func jsonData() throws -> Data +public static func == (a: DecisionQuestionSet, b: DecisionQuestionSet) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct DecisionAnswerSet : Sendable, Hashable, Codable +public struct Example : Sendable, Hashable, Codable +public var input: String +public var answers: [String : String] +public init(input: String, answers: [String : String]) +public static func == (a: DecisionAnswerSet.Example, b: DecisionAnswerSet.Example) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public var examples: [DecisionAnswerSet.Example] +public init(examples: [DecisionAnswerSet.Example]) +public init(csv text: String) throws +public func csv(questionIDs: [String]) -> String +public func problems(for questions: [DecisionQuestion]) -> [String] +public static func answerKey(_ raw: String, for q: DecisionQuestion) -> String? +public static func == (a: DecisionAnswerSet, b: DecisionAnswerSet) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct DecisionQuestion : Sendable, Hashable, Codable +public enum Kind : Sendable, Hashable, Codable +case noul +case choice(criteria: [DecisionQuestion.Criterion]) +case score(levels: [String]) +public static func == (a: DecisionQuestion.Kind, b: DecisionQuestion.Kind) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct Criterion : Sendable, Hashable, Codable +public var key: String +public var description: String? +public init(_ key: String, _ description: String? = nil) +public static func == (a: DecisionQuestion.Criterion, b: DecisionQuestion.Criterion) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public var id: String +public var instructions: String +public var kind: DecisionQuestion.Kind +public init(id: String, instructions: String, kind: DecisionQuestion.Kind) +public static func noul(_ id: String, _ instructions: String) -> DecisionQuestion +public static func choice(_ id: String, _ instructions: String, criteria: KeyValuePairs) -> DecisionQuestion +public static func choice(_ id: String, _ instructions: String, criteria: [String]) -> DecisionQuestion +public static func score(_ id: String, _ instructions: String, criteria: [String]) -> DecisionQuestion +public init(from decoder: any Decoder) throws +public func encode(to encoder: any Encoder) throws +public static func == (a: DecisionQuestion, b: DecisionQuestion) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public enum DecisionState : Sendable, Hashable, Codable +case text(String) +case json(String) +public static func encoding(_ value: T) throws -> DecisionState where T : Encodable +public var rendered: String +public static func == (a: DecisionState, b: DecisionState) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct DecisionRequest : Sendable, Hashable +public var state: DecisionState +public var questions: [DecisionQuestion] +public init(state: DecisionState, questions: [DecisionQuestion]) +public func validate(against limits: DecisionLimits = .jev) throws +public static func == (a: DecisionRequest, b: DecisionRequest) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public struct DecisionLimits : Sendable, Hashable +public var maxQuestions: Int +public var maxChoiceCriteria: Int +public var maxScoreLevels: Int +public var maxStateCharacters: Int? +public init(maxQuestions: Int, maxChoiceCriteria: Int, maxScoreLevels: Int, maxStateCharacters: Int? = nil) +public static let jev: DecisionLimits +public static func == (a: DecisionLimits, b: DecisionLimits) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public struct ChoiceAnswer : Sendable, Hashable, Codable +public var choice: String +public var confidence: Double +public var probabilities: [String : Double] +public init(choice: String, confidence: Double, probabilities: [String : Double]) +public static func == (a: ChoiceAnswer, b: ChoiceAnswer) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct ScoreAnswer : Sendable, Hashable, Codable +public var score: Double +public var confidence: Double +public var probabilities: [Double] +public var legend: [String] +public init(score: Double, confidence: Double, probabilities: [Double], legend: [String]) +public var level: Int +public static func == (a: ScoreAnswer, b: ScoreAnswer) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public enum DecisionAnswer : Sendable, Hashable, Codable +case noul(Double) +case choice(ChoiceAnswer) +case score(ScoreAnswer) +public static func == (a: DecisionAnswer, b: DecisionAnswer) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public enum DecisionFidelity : Sendable, Hashable, Codable +case native +case tokenScored(calibrated: Bool) +public static func == (a: DecisionFidelity, b: DecisionFidelity) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct DecisionUsage : Sendable, Hashable, Codable +public var inputTokens: Int? +public var outputTokens: Int? +public var cost: Double? +public init(inputTokens: Int? = nil, outputTokens: Int? = nil, cost: Double? = nil) +public static func == (a: DecisionUsage, b: DecisionUsage) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public struct Decision : Sendable, Hashable, Codable +public var answers: [String : DecisionAnswer] +public var modelID: ModelID +public var fidelity: DecisionFidelity +public var latency: Duration +public var usage: DecisionUsage? +public init(answers: [String : DecisionAnswer], modelID: ModelID, fidelity: DecisionFidelity, latency: Duration, usage: DecisionUsage? = nil) +public func noul(_ id: String) -> Double? +public func choice(_ id: String) -> ChoiceAnswer? +public func score(_ id: String) -> ScoreAnswer? +public static func == (a: Decision, b: Decision) -> Bool +public func encode(to encoder: any Encoder) throws +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public init(from decoder: any Decoder) throws +public enum DecisionError : Error, LocalizedError, Sendable, Equatable +case invalidRequest(String) +case noRoute(RouteName) +case unavailable(ModelID, ModelAvailability) +case malformedAnswer(String) +case backend(String) +case pairing(String) +public var errorDescription: String? +public static func == (a: DecisionError, b: DecisionError) -> Bool public protocol DownloadableModelProvider : ModelProvider public var residencyEventStream: AsyncStream? public struct InstalledModel : Sendable, Hashable, Codable, Identifiable @@ -673,6 +913,39 @@ public static func == (a: HostTranscript.ReplyInProgress, b: HostTranscript.Repl public enum JSONPath public static func value(at path: String, in root: Any) -> Any? public static func stringify(_ value: Any?) -> String +@MainActor final public func decide(route: RouteName, state: DecisionState, questions: [DecisionQuestion]) async throws -> Decision +public struct DecisionEvaluation : Sendable +public struct QuestionScore : Sendable, Hashable +public var id: String +public var kind: DecisionCalibration.Sample.Kind +public var marked: Int +public var correct: Int +public var accuracy: Double +public var meanConfidenceWhenRight: Double? +public var meanConfidenceWhenWrong: Double? +public static func == (a: DecisionEvaluation.QuestionScore, b: DecisionEvaluation.QuestionScore) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public struct Miss : Sendable, Hashable +public var input: String +public var questionID: String +public var expected: String +public var answer: String +public var confidence: Double +public static func == (a: DecisionEvaluation.Miss, b: DecisionEvaluation.Miss) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public var modelID: ModelID +public var fidelity: DecisionFidelity +public var questions: [DecisionEvaluation.QuestionScore] +public var misses: [DecisionEvaluation.Miss] +public var samples: [DecisionCalibration.Sample] +public var decisions: Int +public var totalLatency: Duration +public var accuracy: Double +public var calibration: DecisionCalibration.Measurement +public func score(_ questionID: String) -> DecisionEvaluation.QuestionScore? +@MainActor final public func evaluate(route: RouteName, questions: [DecisionQuestion], answerSet: DecisionAnswerSet, progress: (@MainActor @Sendable (Int, Int) -> Void)? = nil) async throws -> DecisionEvaluation @MainActor final public func makeSession(route: RouteName, tools: [any Tool] = [], instructions: String? = nil, restoring transcript: Transcript? = nil, includeMCPTools: Bool = true, mcpAppHints: Bool = false, options: SessionOptions = .init(), authorizer: (any ToolCallAuthorizer)? = nil) throws -> LocalLMLabSession @MainActor final public class LocalLMLab public struct Configuration @@ -699,10 +972,11 @@ public var version: Int public var routes: [RouteName : ModelID] public var residency: ModelResidency public var installedModels: [InstalledModel] -public init(version: Int = LocalLMLabState.currentVersion, routes: [RouteName : ModelID] = [:], residency: ModelResidency = .lastUsedOnly, installedModels: [InstalledModel] = []) +public var decisionRoutes: [RouteName : ModelID] +public init(version: Int = LocalLMLabState.currentVersion, routes: [RouteName : ModelID] = [:], residency: ModelResidency = .lastUsedOnly, installedModels: [InstalledModel] = [], decisionRoutes: [RouteName : ModelID] = [:]) +public init(from decoder: any Decoder) throws public static func == (a: LocalLMLabState, b: LocalLMLabState) -> Bool public func encode(to encoder: any Encoder) throws -public init(from decoder: any Decoder) throws @MainActor public struct ConnectorsFacade @MainActor public func isAuthorized(_ connector: Connector) -> Bool @MainActor public func requestAccess(_ connector: Connector) async -> ConnectorAccessResult @@ -1261,9 +1535,18 @@ public func prewarm(_ id: ModelID) async public var advertisedModels: [ModelID] public func turnArtifacts(in transcript: Transcript) -> TurnArtifacts public func makeSession(for id: ModelID, tools: [any Tool], instructions: String?, transcript: Transcript?, options: SessionOptions) throws -> LanguageModelSession +@MainActor final public func register(decision provider: any DecisionProvider) throws +@MainActor final public func replace(decision provider: any DecisionProvider) +@MainActor final public func removeDecisionProvider(scheme: String) +@MainActor final public func route(decision route: RouteName, to id: ModelID) +@MainActor final public func modelID(forDecision route: RouteName) -> ModelID? +@MainActor final public func decisionProvider(for id: ModelID) -> (any DecisionProvider)? +@MainActor final public func decisionAvailability(for id: ModelID) -> ModelAvailability @MainActor final public class ModelRegistry @MainActor final public var providers: [any ModelProvider] @MainActor final public var routes: [RouteName : ModelID] +@MainActor final public var decisionProviders: [any DecisionProvider] +@MainActor final public var decisionRoutes: [RouteName : ModelID] @MainActor final public var residency: ModelResidency @MainActor final public let events: AsyncStream @MainActor public init() @@ -1549,7 +1832,8 @@ public var minP: Double? public var repetitionPenalty: Double? public var repetitionContextSize: Int? public var prefillStepSize: Int? -public init(webSearch: Bool = false, webSearchMaxUses: Int? = nil, webSearchAllowedDomains: [String] = [], webSearchBlockedDomains: [String] = [], effort: Effort? = nil, temperature: Double? = nil, topP: Double? = nil, maxOutputTokens: Int? = nil, userLocation: UserLocation? = nil, seed: UInt64? = nil, topK: Int? = nil, minP: Double? = nil, repetitionPenalty: Double? = nil, repetitionContextSize: Int? = nil, prefillStepSize: Int? = nil) +public var maxModelCallsPerTurn: Int? +public init(webSearch: Bool = false, webSearchMaxUses: Int? = nil, webSearchAllowedDomains: [String] = [], webSearchBlockedDomains: [String] = [], effort: Effort? = nil, temperature: Double? = nil, topP: Double? = nil, maxOutputTokens: Int? = nil, userLocation: UserLocation? = nil, seed: UInt64? = nil, topK: Int? = nil, minP: Double? = nil, repetitionPenalty: Double? = nil, repetitionContextSize: Int? = nil, prefillStepSize: Int? = nil, maxModelCallsPerTurn: Int? = nil) public var serverTools: Set public func merged(onto base: SessionOptions) -> SessionOptions public static func == (a: SessionOptions, b: SessionOptions) -> Bool @@ -2049,7 +2333,9 @@ public init(source: AdapterSource) public static func == (a: AdapterSpec, b: AdapterSpec) -> Bool public struct MLXModelProvider : DownloadableModelProvider public static var scheme: String -public init(cacheDirectory: URL? = nil, residentModelLimit: Int = 1, preflightLimits: MLXPreflightLimits = MLXPreflightLimits(), pinnedRevisions: [String : String] = [:], supplyChainPolicy: MLXSupplyChainPolicy = .default, pinStore: (any MLXPinStore)? = nil, managedPinStore: (any MLXManagedPinStore)? = nil) +public let maxModelCallsPerTurn: Int +public static let defaultMaxModelCallsPerTurn: Int +public init(cacheDirectory: URL? = nil, residentModelLimit: Int = 1, preflightLimits: MLXPreflightLimits = MLXPreflightLimits(), pinnedRevisions: [String : String] = [:], supplyChainPolicy: MLXSupplyChainPolicy = .default, pinStore: (any MLXPinStore)? = nil, managedPinStore: (any MLXManagedPinStore)? = nil, maxModelCallsPerTurn: Int = MLXModelProvider.defaultMaxModelCallsPerTurn) public var residencyEventStream: AsyncStream? public func effectivePin(for repoID: String) -> MLXPin? public func buildTimePin(for repoID: String) -> String? @@ -2127,12 +2413,36 @@ public struct MLXPreflightLimits : Sendable, Equatable public var maxWeightFractionOfRAM: Double public init(maxWeightFractionOfRAM: Double = 0.7) public static func == (a: MLXPreflightLimits, b: MLXPreflightLimits) -> Bool +public struct MLXPrefillActivity : Sendable, Equatable +public let repoID: String +public let promptTokens: Int +public let modelCall: Int +public let startedAt: Date +public init(repoID: String, promptTokens: Int, modelCall: Int, startedAt: Date) +public static func == (a: MLXPrefillActivity, b: MLXPrefillActivity) -> Bool +public enum MLXPrefillMonitor +public static var active: [MLXPrefillActivity] public struct MLXSupplyChainPolicy : Sendable public var verification: MLXWeightVerification public var trustPolicy: (any MLXModelTrustPolicy)? public var cacheLimits: MLXCacheLimits public init(verification: MLXWeightVerification = .default, trustPolicy: (any MLXModelTrustPolicy)? = nil, cacheLimits: MLXCacheLimits = .default) public static let `default`: MLXSupplyChainPolicy +public enum MLXUsageMetadataKey +public static let promptTokens: String +public static let generatedTokens: String +public static let promptSeconds: String +public static let generateSeconds: String +public static let promptTokensPerSecond: String +public static let tokensPerSecond: String +public static let rounds: String +public static let stopReason: String +public static let draftTokensProposed: String +public static let draftTokensAccepted: String +public static let speculativeRounds: String +public static let residentMemoryBytes: String +public static let peakMemoryBytes: String +public static let gpuMemoryLimitBytes: String public enum MLXWeightVerification : Sendable, Equatable case enabled case disabled @@ -2149,11 +2459,75 @@ public var filename: String public var reason: MLXWeightVerificationError.Reason public init(filename: String, reason: MLXWeightVerificationError.Reason) public var errorDescription: String? +public struct OpenJevWrapper : Sendable, Hashable +public var system: String +public var inputLabel: String +public init(system: String, inputLabel: String) +public static let `default`: OpenJevWrapper +public static func == (a: OpenJevWrapper, b: OpenJevWrapper) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public enum OpenJevWarning : Sendable, Hashable +case mixtureOfExperts +public static func == (a: OpenJevWarning, b: OpenJevWarning) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public struct OpenJevDiagnostics : Sendable +public struct Question : Sendable +public let labels: [String] +public let prompt: String +public let labelMass: Double +public let latency: Duration +public let questions: [String : OpenJevDiagnostics.Question] +public let sharedPrefixTokens: Int +public struct OpenJevDecisionProvider : DecisionProvider +public let scheme: String +public let mlx: MLXModelProvider +public let wrapper: OpenJevWrapper +public let calibration: DecisionCalibration? +public let tuning: DecisionQuestionSet.Tuning? +public init(mlx: MLXModelProvider, wrapper: OpenJevWrapper = .default, calibration: DecisionCalibration? = nil, scheme: String = "openjev") +public init(mlx: MLXModelProvider, tunedWith set: DecisionQuestionSet, scheme: String = "openjev") +public func tuningMismatch(for id: ModelID) -> String? +public var limits: DecisionLimits +public func fidelity(for id: ModelID) -> DecisionFidelity +public func availability(for id: ModelID) -> ModelAvailability +public func warnings(for id: ModelID) -> [OpenJevWarning] +public func prewarm(_ id: ModelID) async +public func decide(_ request: DecisionRequest, using id: ModelID) async throws -> Decision +public func decideWithDiagnostics(_ request: DecisionRequest, using id: ModelID) async throws -> (decision: Decision, diagnostics: OpenJevDiagnostics) +@MainActor public func pair(decider: ModelID, generator: ModelID, registry: ModelRegistry) async throws -> DecisionPairing ``` ## LocalLMLabSDKRemote ```swift +public struct JevProviderConfig : Sendable, Equatable +public var scheme: String +public var displayName: String +public var endpoint: URL +public var auth: RemoteProviderConfig.Auth +public var models: [String] +public var allowArbitraryModelIDs: Bool +public var fidelity: DecisionFidelity +public var limits: DecisionLimits +public var responseLimits: RemoteResponseLimits +public var timeout: TimeInterval +public init(scheme: String, displayName: String, endpoint: URL, auth: RemoteProviderConfig.Auth, models: [String] = [], allowArbitraryModelIDs: Bool = true, fidelity: DecisionFidelity, limits: DecisionLimits = .jev, responseLimits: RemoteResponseLimits = .default, timeout: TimeInterval = 60) +public static func openRouter(key: String) -> JevProviderConfig +public static func featherless(key: String) -> JevProviderConfig +public static func featherlessDemo() -> JevProviderConfig +public func validated() throws -> JevProviderConfig +public static func == (a: JevProviderConfig, b: JevProviderConfig) -> Bool +public struct JevDecisionProvider : DecisionProvider +public let config: JevProviderConfig +public init(_ config: JevProviderConfig) +public var scheme: String +public var limits: DecisionLimits +public func fidelity(for id: ModelID) -> DecisionFidelity +public func availability(for id: ModelID) -> ModelAvailability +public func decide(_ request: DecisionRequest, using id: ModelID) async throws -> Decision +public static func requestBody(model: String, request: DecisionRequest) throws -> Data public static func openAI(apiKey: String, models: [RemoteModel] = []) -> RemoteProviderConfig public static func anthropic(apiKey: String, models: [RemoteModel] = []) -> RemoteProviderConfig public static func openAIResponses(apiKey: String, models: [RemoteModel] = []) -> RemoteProviderConfig diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md index 7d63572..ab9bf4a 100644 --- a/docs/migrating-to-2.0.md +++ b/docs/migrating-to-2.0.md @@ -136,6 +136,18 @@ passed `restoring:` before `tools:`). Declare support with `MCPSettings(handlers: MCPClientHandlers().advertisingMCPApps())`. Reference app: [`examples/mcp-chat`](../examples/mcp-chat/); walkthrough [`sdk-guide.md` §3f](sdk-guide.md#3f-mcp-apps-showing-a-servers-interactive-views). +- **Decision models (Jev).** `lab.decide(route:state:questions:)` asks typed questions (yes / no, + one of several, a scale) and gets one allowed answer per question with probabilities, no text. + The same call works with TypeSafe's Jev via OpenRouter and Featherless's Simple Jev + (`JevDecisionProvider`, `LocalLMLabSDKRemote`) or **OpenJev** on a local MLX model + (`OpenJevDecisionProvider`, `LocalLMLabSDKInference`, macOS 27). `lab.models.pair(decision:generator:)` + keeps a small decider and your chat model loaded together. `DecisionQuestionSet`, + `DecisionAnswerSet` and `lab.evaluate` carry tested questions into your app and score them in + CI. Decision routes are separate from chat routes and are saved by `lab.snapshot()` + (`LocalLMLabState` gains `decisionRoutes`, defaulted, so existing call sites and saved snapshots + are unaffected). Introduction: [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html); + API: [`sdk-guide.md` §6c](sdk-guide.md#6c-decision-models-jev--labdecide); playground: + [`examples/jevdk`](../examples/jevdk/). - **Local (MLX) model usage, speed and memory per turn**: `MLXUsageMetadataKey` on the response's `usage.metadata` (tokens, pp / tg, model calls, MLX memory), and `MLXPrefillMonitor` for "what is the model reading right now" while nothing has appeared yet. diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index b6a3833..8dd68b1 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -46,6 +46,7 @@ deeply. Ordered smallest → largest: | A CLI coding agent — two models with routing, workspace + host `Process` tools, MCP | [`code-buddy`](../examples/code-buddy) | 298 | | An MCP dataset pulled through a mechanical SQL pipeline instead of the model copying rows | [`aiql`](../examples/aiql) | 381 | | An MLX model shipped pinned to an exact commit, plus checking a small model really called its tool | [`vistanova`](../examples/vistanova) | 931 (whole app) | +| Decision models (Jev): design and test typed questions on a local MLX model and hosted Jev, then export them for your app ([§6c](#6c-decision-models-jev--labdecide)) | [`jevdk`](../examples/jevdk) | ~3,000 (whole app) | Line counts are from [`annotated-examples.md`](annotated-examples.md#how-much-code-is-this-really), which also has the @@ -1741,6 +1742,211 @@ plain `RemoteProviderDraft` / `ProviderTestOutcome` values, so a macOS-26 host a panel and hand the actual `RemoteModelProvider` work to a 27-only helper. → *the full pattern is [`examples/model-switch`](../examples/model-switch/).* +## 6c. Decision models (Jev) — `lab.decide` + +> **Reach for this when** your app needs a small, fixed decision about some text — which route, +> which team, does this need the web, is this sensitive — and you'd otherwise ask a chat model and +> parse its reply. +> +> **New to decision models?** They behave nothing like a chatbot. Read +> [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html) first (what they +> are, for people who know chatbots, with links to TypeSafe's and others' docs), then +> [JevDK's developer's guide](../examples/jevdk/GUIDE.md) for the workflow. +> +> **Examples that use it:** [`jevdk`](../examples/jevdk/). + +A **decision model** (a *decider*) takes one input and a few typed questions, and answers each +with one of the options you allowed plus a probability for every option. It writes no text. +TypeSafe's **Jev** is the original; the SDK uses Jev's vocabulary and asks the same questions of +three backends: + +| Decider | Runs | Module | OS | `Decision.fidelity` | +|---|---|---|---|---| +| **OpenJev** (`OpenJevDecisionProvider`) | On the Mac, on an MLX model | Inference | macOS 27 | `.tokenScored(calibrated:)` | +| **TypeSafe Jev** via OpenRouter (`JevDecisionProvider(.openRouter(key:))`) | Hosted | Remote | macOS 26+ | `.native` | +| **Featherless Simple Jev** (`.featherless(key:)`, `.featherlessDemo()`) | Hosted | Remote | macOS 26+ | `.tokenScored(calibrated: false)` | + +`lab.decide` and everything else in this section except OpenJev is in Core and works on macOS 26. + +### Register a decider and route to it + +Deciders are registered and routed separately from chat models (they never appear in a chat +model picker), with their own scheme namespace: + +```swift +import LocalLMLabSDKInference // OpenJev +import LocalLMLabSDKRemote // hosted Jev + +try lab.models.register(decision: OpenJevDecisionProvider(mlx: mlx)) // the MLXModelProvider you already use +lab.models.route(decision: "support", to: ModelID("openjev:mlx-community/Qwen3-4B-4bit")!) + +// or hosted — off until you register one; questions and input are then sent to that service +try lab.models.register(decision: JevDecisionProvider(.openRouter(key: key))) +lab.models.route(decision: "support", to: ModelID("openrouter:typesafe/jev-1.13")!) +``` + +`ModelRegistry` also has `replace(decision:)`, `removeDecisionProvider(scheme:)`, +`modelID(forDecision:)`, `decisionProvider(for:)` and `decisionAvailability(for:)` (a +`ModelAvailability`, e.g. not downloaded or needs a key). Decision routes are saved by +`lab.snapshot()` like chat routes. + +### Ask + +```swift +let d = try await lab.decide(route: "support", state: .text(message), questions: [ + .choice("team", "Which team should handle this message?", criteria: [ + "billing": "payments, invoices and refunds", + "technical": "bugs, outages and product errors", + "account": "profile, login and subscription changes", + ]), + .noul("refund", "Does the customer explicitly ask for money back?"), + .score("urgency", "How urgent is it?", criteria: ["can wait", "soon", "now"]), +]) + +d.choice("team") // ChoiceAnswer: .choice "technical", .confidence, .probabilities ["billing": …, …] +d.noul("refund") // Double: probability of yes +d.score("urgency") // ScoreAnswer: .score 1.6 (expected level, 0-based), .level, .probabilities, .legend +d.fidelity // .native or .tokenScored(calibrated:) +d.modelID, d.latency, d.usage // usage: tokens and cost, from hosted backends +``` + +- **Question kinds** use Jev's names: `noul` (yes / no; the answer is P(yes)), `choice` (one of + several, each with an optional description), `score` (an ordered scale described in words). + Ids are what your code reads; `instructions` is what the model reads. +- **The input** is `state`: `.text(String)`, or `try .encoding(someEncodable)` to send JSON. For + a conversation, send the last few turns, not the whole history. +- **There is no system prompt.** Hosted Jev has none, so context there never reaches it; put + anything a question needs in `state` or in the question. Then the same questions behave the same + on every backend. +- **Validated before anything is sent** (unique ids, at least two options, the backend's + `DecisionLimits`) and checked after (every question answered, the right kind, probabilities in + 0–1). Errors are `DecisionError`: `.invalidRequest`, `.noRoute`, `.unavailable`, + `.malformedAnswer`, `.backend` (a hosted service's own message), `.pairing`. +- **Input size.** `DecisionLimits.maxStateCharacters` makes an over-long input fail before it's + sent, telling you to shorten it. The Featherless demo preset sets 6,000 characters. +- **One route per call.** To ask some questions locally and others hosted, make two calls; the + split (and the order: a local privacy question first, say) is your app's policy. + +### Confidence, fidelity and calibration + +`.native` probabilities come from a model trained to make decisions (TypeSafe). `.tokenScored` +ones are read off an ordinary language model — how likely it was to say each option's label — +and on a small model they're often 100% even when it's wrong. Don't put tight thresholds on +uncalibrated `.tokenScored` answers. + +For a local decider you fit a **calibration** on your own marked answers (JevDK's **Calibrate…** +does it) and pass it in; `fidelity` then says `.tokenScored(calibrated: true)`. It holds for one +model, revision, wrapper and kind of question, so the SDK ships no fitted values. + +```swift +let cal = DecisionCalibration.fit(samples) // [DecisionCalibration.Sample], from marked answers +cal.measure(samples) // accuracy, expectedCalibrationError, logLoss +let openjev = OpenJevDecisionProvider(mlx: mlx, calibration: cal) +``` + +Calibration is temperature scaling per question kind: it never changes which answer wins, only +how sure it says it is. + +### OpenJev: the local decider + +- **How it decides:** one forward pass per question, no generation. Each option gets a + single-token label (`A`–`Z`, `0`–`9`, `Yes`/`No`) and OpenJev reads the probability of each + label. The input is processed once and shared by all the questions, so four questions on Qwen3 + 4B take about 150 ms. That's also why its limits are smaller than hosted Jev's: up to 26 choice + options, 10 score levels, 64 questions a call. +- **The wrapper.** Each question goes to the model inside a fixed prompt (`OpenJevWrapper.default`: + *"You answer questions about a piece of text…"*, the input labelled `Text`), chosen with the + SDK's evaluation set. You can pass your own `OpenJevWrapper`, but the wording moves answers as + much as your questions do, so change it only with your answer set to check against. +- **Which model.** A small **dense** instruct model: Qwen3-4B-4bit is the tested default (2.6 GB). + Qwen3-1.7B is too weak. **Not a mixture-of-experts model**: an MoE model's probabilities + shifted with how the prompt was split. `openjev.warnings(for:)` returns + `.mixtureOfExperts` for one (a warning, not a block). +- **Only models the SDK downloaded count.** Availability follows `MLXModelProvider`: a copy + another tool put in the Hugging Face cache reads as not downloaded until the SDK has verified it. +- **Debugging:** `decideWithDiagnostics(_:using:)` also returns each question's exact prompt, + how much probability fell on the allowed labels (`labelMass`; low means the model wanted to + answer something else, so reword the question) and its timing. + +### Hosted Jev + +`JevProviderConfig` presets set the endpoint, scheme, fidelity, limits and model list: +`.openRouter(key:)` (TypeSafe's `typesafe/jev-1.13`; `typesafe/jev-router` picks a model for +you), `.featherless(key:)` and `.featherlessDemo()` (free, rate-limited, no key). +`validated()` applies the chat providers' checks (https, no line breaks in keys). Questions and +choice options keep your order on the wire, JSON `state` is sent as JSON, and `d.usage` carries +the tokens and cost. Like other remote providers, it's off by default and your UI should say +where inputs are sent. + +### A decider and your chat model: one model or two + +The pattern the SDK is built for is two models per feature: a decider for the typed choices (does +this need the web, which tool, is it sensitive) and a chat model for the text, both on the Mac. + +- **Share one model.** Route OpenJev and the chat route to the same dense MLX model: one copy in + memory. The low-memory option; a big chat model makes a slower decider than a small one. +- **Two models resident.** `MLXModelProvider` keeps one local model loaded by default + (`residentModelLimit: 1`), so a decider asked every turn would evict the chat model. Pair them: + + ```swift + lab.models.route(decision: "triage", to: ModelID("openjev:mlx-community/Qwen3-4B-4bit")!) + lab.models.route("writer", to: ModelID("mlx:mlx-community/Qwen3.8-27B-4bit")!) + let pairing = try await lab.models.pair(decision: "triage", generator: "writer") + // DecisionPairing: deciderBytes, generatorBytes, budgetBytes, residentModels, note + ``` + + `pair(decision:generator:)` checks the two models' weights together against the memory budget + (`maxWeightFractionOfRAM`, 70% of RAM by default), raises the model cache's capacity to hold + both (never lowers it) and loads them. Doesn't fit: `DecisionError.pairing` with the numbers, + and nothing changes. A shared model counts once (and is refused if it's MoE); a hosted decider + or a non-MLX chat model (Apple's on-device model) needs only one. On a 64 GB Mac, pairing a + Qwen3 4B decider with Qwen3.8 27B took 2.3 s, and the next decision ran in 63 ms with no reload. + +- **Still in progress:** keeping the pair loaded under memory pressure (the cache can still + unload all but the most recent model then); a measured pair on a 16 GB Mac; and loading + text-only uses of vision-capable models with the faster text implementation. + +### From JevDK to your app: question sets, answer sets, `lab.evaluate` + +Questions are the work, so the SDK carries them from testing to shipping intact: + +```swift +// The questions exactly as tested in JevDK (File → Export for App…), with the model, revision, +// wrapper and calibration they were tested with. +let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "support.decisions", withExtension: "json")!) +let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // the tested wrapper + calibration +try lab.models.register(decision: openjev) +let decider = set.tuning?.model ?? ModelID("openjev:mlx-community/Qwen3-4B-4bit")! +lab.models.route(decision: "support", to: decider) +if let why = openjev.tuningMismatch(for: decider) { log(why) } // a different model or revision on this Mac +let d = try await lab.decide(route: "support", state: .text(message), questions: set.questions) + +// In CI: score any decider against your answer set (CSV: input, then one column per question id). +let answers = try DecisionAnswerSet(csv: String(contentsOf: answersURL, encoding: .utf8)) +let result = try await lab.evaluate(route: "support", questions: set.questions, answerSet: answers) +#expect(result.score("refund")!.accuracy >= 0.95) +``` + +- `DecisionQuestionSet` is JSON in Jev's question shape (`{id, type, instructions, criteria}`); + a file from a newer SDK, or with questions that couldn't be asked, is refused. +- `DecisionAnswerSet` matches answers leniently (yes/no as Yes, No, true, false, 1, 0; a choice + by key; a score by level number or text); `problems(for:)` lists cells that don't fit, and + `evaluate` refuses such a set rather than skipping rows. +- `DecisionEvaluation` gives accuracy per question, the misses, mean confidence when right and + when wrong (a big gap means a threshold can catch the misses), calibration error, and + `samples` for `DecisionCalibration.fit`. + +### Writing questions that work + +- Only ask what a model can judge from the input and general knowledge — **never current + facts**. "Does this user query need live data?" is fine; "has this changed recently?" isn't. +- Ask about **what the text says**: "Does the customer *explicitly* ask for money back?" beats + "Does the customer want a refund?" +- Prefer noul and choice; small local models are weakest on score scales. +- Give choice options short descriptions, not just names. +- Check every wording on the model you'll ship, against your answer set. Different models + prefer different wordings. + ## 7. Connectors: Calendar, Reminders, Contacts, Location > **Reach for these when** your app's value is "the model can see — or change — my calendar, @@ -3097,6 +3303,120 @@ enum LocalLMLabSDKVersion { static let current: String } // "1.0.0-beta.N", "1 a separate xcframework on the same release — [§1a](#1a-targeting-macos-26-and-macos-27-from-one-build), [§6b](#6b-online-providers--gpt-claude-online-openrouter-locallmlabsdkremote)); `ModelPickerView` / `ClaudeAuthField` / `AIModelsSettingsView` are in **`LocalLMLabSDKComponents`** ([§11](#11-components-prebuilt-swiftui-mcp-servers--the-model-layer)). +### Decision models (`lab.decide`, OpenJev, hosted Jev) + +Walkthrough: [§6c](#6c-decision-models-jev--labdecide). Core unless marked; `OpenJev*` is +**`LocalLMLabSDKInference`** (macOS 27), `Jev*` is **`LocalLMLabSDKRemote`**. + +```swift +// --- asking ----------------------------------------------------------------- +extension LocalLMLab { + func decide(route: RouteName, state: DecisionState, questions: [DecisionQuestion]) async throws -> Decision + func evaluate(route: RouteName, questions: [DecisionQuestion], answerSet: DecisionAnswerSet, + progress: (@MainActor (Int, Int) -> Void)? = nil) async throws -> DecisionEvaluation +} +struct DecisionQuestion: Sendable, Hashable, Codable { // Codable as Jev's {id, type, instructions, criteria} + enum Kind { case noul, choice(criteria: [Criterion]), score(levels: [String]) } + struct Criterion { var key: String; var description: String?; init(_ key: String, _ description: String? = nil) } + var id: String; var instructions: String; var kind: Kind + static func noul(_ id: String, _ instructions: String) -> DecisionQuestion + static func choice(_ id: String, _ instructions: String, criteria: KeyValuePairs) -> DecisionQuestion + static func choice(_ id: String, _ instructions: String, criteria: [String]) -> DecisionQuestion + static func score(_ id: String, _ instructions: String, criteria: [String]) -> DecisionQuestion +} +enum DecisionState { case text(String), json(String); static func encoding(_ value: T) throws -> DecisionState } +struct DecisionRequest { var state: DecisionState; var questions: [DecisionQuestion]; func validate(against: DecisionLimits = .jev) throws } +struct DecisionLimits { var maxQuestions, maxChoiceCriteria, maxScoreLevels: Int; var maxStateCharacters: Int?; static let jev } + +// --- answers ---------------------------------------------------------------- +struct Decision: Sendable, Hashable, Codable { + var answers: [String: DecisionAnswer] + var modelID: ModelID; var fidelity: DecisionFidelity; var latency: Duration; var usage: DecisionUsage? + func noul(_ id: String) -> Double? // P(yes) + func choice(_ id: String) -> ChoiceAnswer? + func score(_ id: String) -> ScoreAnswer? +} +enum DecisionAnswer { case noul(Double), choice(ChoiceAnswer), score(ScoreAnswer) } +struct ChoiceAnswer { var choice: String; var confidence: Double; var probabilities: [String: Double] } +struct ScoreAnswer { var score: Double; var confidence: Double; var probabilities: [Double]; var legend: [String]; var level: Int } +enum DecisionFidelity { case native, tokenScored(calibrated: Bool) } +struct DecisionUsage { var inputTokens: Int?; var outputTokens: Int?; var cost: Double? } +enum DecisionError: Error { case invalidRequest(String), noRoute(RouteName), unavailable(ModelID, ModelAvailability), + malformedAnswer(String), backend(String), pairing(String) } + +// --- providers and routes --------------------------------------------------- +protocol DecisionProvider: Sendable { // implement your own backend + var scheme: String { get } + var limits: DecisionLimits { get } // default: .jev + func fidelity(for id: ModelID) -> DecisionFidelity + func availability(for id: ModelID) -> ModelAvailability + func decide(_ request: DecisionRequest, using id: ModelID) async throws -> Decision + func owns(_ id: ModelID) -> Bool // default: id.scheme == scheme + func prewarm(_ id: ModelID) async // default: nothing +} +extension ModelRegistry { + func register(decision: any DecisionProvider) throws; func replace(decision: any DecisionProvider) + func removeDecisionProvider(scheme: String) + func route(decision: RouteName, to: ModelID); func modelID(forDecision: RouteName) -> ModelID? + func decisionProvider(for: ModelID) -> (any DecisionProvider)?; func decisionAvailability(for: ModelID) -> ModelAvailability + func pair(decision: RouteName, generator: RouteName) async throws -> DecisionPairing +} +struct DecisionPairing { var decider, generator: ModelID; var deciderBytes, generatorBytes, budgetBytes: Int64? + var residentModels: Int; var note: String? } +protocol PairableDecisionProvider: DecisionProvider { /* a local decider that can stay resident with a generator */ } + +// --- OpenJev (LocalLMLabSDKInference, macOS 27) -------------------------------- +struct OpenJevDecisionProvider: PairableDecisionProvider { // scheme "openjev"; limits 64 questions, 26 options, 10 levels + init(mlx: MLXModelProvider, wrapper: OpenJevWrapper = .default, calibration: DecisionCalibration? = nil, scheme: String = "openjev") + init(mlx: MLXModelProvider, tunedWith set: DecisionQuestionSet, scheme: String = "openjev") + let wrapper: OpenJevWrapper; let calibration: DecisionCalibration?; let tuning: DecisionQuestionSet.Tuning? + func warnings(for id: ModelID) -> [OpenJevWarning] // .mixtureOfExperts + func tuningMismatch(for id: ModelID) -> String? // not the model / revision the set was tested with + func decideWithDiagnostics(_ request: DecisionRequest, using id: ModelID) async throws -> (decision: Decision, diagnostics: OpenJevDiagnostics) +} +struct OpenJevWrapper { var system: String; var inputLabel: String; static let `default` } +struct OpenJevDiagnostics { let questions: [String: Question]; let sharedPrefixTokens: Int } // Question: labels, prompt, labelMass, latency + +// --- hosted Jev (LocalLMLabSDKRemote) ------------------------------------------- +struct JevDecisionProvider: DecisionProvider { init(_ config: JevProviderConfig) } +struct JevProviderConfig { // scheme, displayName, endpoint, auth, models, allowArbitraryModelIDs, fidelity, limits, responseLimits, timeout + static func openRouter(key: String) -> JevProviderConfig // TypeSafe Jev, .native + static func featherless(key: String) -> JevProviderConfig // .tokenScored(calibrated: false) + static func featherlessDemo() -> JevProviderConfig // no key, rate-limited, maxStateCharacters 6000 + func validated() throws -> JevProviderConfig +} + +// --- calibration, question sets, answer sets, evaluation --------------------------- +struct DecisionCalibration: Codable { + var noulTemperature, choiceTemperature, scoreTemperature: Double; static let none + struct Sample { var kind: Kind; var probabilities: [Double]; var correct: [Int] } // Kind: .noul, .choice, .score + struct Measurement { var count: Int; var accuracy, expectedCalibrationError, logLoss: Double } + static func fit(_ samples: [Sample]) -> DecisionCalibration + func measure(_ samples: [Sample]) -> Measurement + func apply(_ probabilities: [Double], kind: Sample.Kind) -> [Double] +} +struct DecisionQuestionSet: Codable { // JevDK's "Export for App…" + static let currentFormat: Int + var format: Int; var name: String; var questions: [DecisionQuestion]; var tuning: Tuning? + struct Tuning { var model: ModelID?; var revision, system, inputLabel: String?; var calibration: DecisionCalibration?; var testedAt: Date? } + init(name: String, questions: [DecisionQuestion], tuning: Tuning? = nil) + init(contentsOf url: URL) throws; init(json: Data) throws; func jsonData() throws -> Data +} +struct DecisionAnswerSet: Codable { // CSV: input, then one column per question id + struct Example { var input: String; var answers: [String: String] } + var examples: [Example] + init(csv: String) throws; func csv(questionIDs: [String]) -> String + func problems(for questions: [DecisionQuestion]) -> [String] + static func answerKey(_ raw: String, for q: DecisionQuestion) -> String? +} +struct DecisionEvaluation { + var modelID: ModelID; var fidelity: DecisionFidelity; var questions: [QuestionScore]; var misses: [Miss] + var samples: [DecisionCalibration.Sample]; var decisions: Int; var totalLatency: Duration + var accuracy: Double; var calibration: DecisionCalibration.Measurement + func score(_ questionID: String) -> QuestionScore? // marked, correct, accuracy, meanConfidenceWhenRight / WhenWrong +} +``` + ### Tool authority (`ToolCallAuthorizer`, `ToolImpact`, confirmation) The two levers from [§7c](#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first), as types. Everything here is in `LocalLMLabSDKCore` except the last From 0c4751d5057cc38f6d3889c2011c451c7cbd8fa0 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:00:51 -0700 Subject: [PATCH 31/73] JevDK: hosted results show A, B, C labels like local ones Hosted runs have no labels of their own, so the result rows showed each option's key in the narrow label column, where it wrapped a letter group per line ("bil / lin / g") and repeated the option name beside it. They now use the same position labels as a local run, so the two backends' rows line up, and the column never wraps. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/Sources/JevDK/ResultsView.swift | 1 + examples/jevdk/Sources/OpenJevKit/EditorModel.swift | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/examples/jevdk/Sources/JevDK/ResultsView.swift b/examples/jevdk/Sources/JevDK/ResultsView.swift index 67c28a1..627a22d 100644 --- a/examples/jevdk/Sources/JevDK/ResultsView.swift +++ b/examples/jevdk/Sources/JevDK/ResultsView.swift @@ -256,6 +256,7 @@ struct BackendAnswer: View { HStack(spacing: 8) { Text(result.labels[i]) .font(.system(size: 12, design: .monospaced).weight(.bold)) + .lineLimit(1) .frame(width: 28, alignment: .leading) Text(optionText(i)) .font(.jCaption) diff --git a/examples/jevdk/Sources/OpenJevKit/EditorModel.swift b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift index 0c2e139..bdd20fb 100644 --- a/examples/jevdk/Sources/OpenJevKit/EditorModel.swift +++ b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift @@ -217,8 +217,11 @@ public struct QuestionResult: Identifiable, Sendable { default: return nil } + // Hosted runs have no labels of their own (they answer with the keys); show the same + // position labels a local run does (A, B, … for a choice) so the two line up. + let positional = q.kind == .choice ? keys.indices.map { String(UnicodeScalar(65 + min($0, 25))!) } : keys return QuestionResult( - question: q, keys: keys, labels: diag?.labels ?? keys, descriptions: descs, probabilities: probs, + question: q, keys: keys, labels: diag?.labels ?? positional, descriptions: descs, probabilities: probs, rawProbabilities: probs, confidence: conf, labelMass: diag?.labelMass ?? 1, prompt: diag?.prompt ?? prompt, milliseconds: diag?.milliseconds ?? perQuestionMS, fidelity: d.fidelity) } From cd97a99b779a9a2ecae8e5be9f2749147218197b Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:30:35 -0700 Subject: [PATCH 32/73] JevDK: Dock icon "Jdk" in white on LocalLM Lab's blue rounded square, the same design as the LocalLM Lab app icon. Drawn in code and set as applicationIconImage at launch, so `swift run` shows it too. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/Sources/JevDK/JevDKApp.swift | 30 +++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index a87cbf6..d58d2ec 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -31,6 +31,8 @@ struct JevDKApp: App { if NSApplication.shared.activationPolicy() != .regular { NSApplication.shared.setActivationPolicy(.regular) } + // The Dock icon, drawn in code: `swift run` has no bundle to hold one. + NSApplication.shared.applicationIconImage = AppIcon.image() DispatchQueue.main.async { NSApplication.shared.activate() } } @@ -348,3 +350,31 @@ struct BackendsSheet: View { .font(.jBody) } } + +/// JevDK's Dock icon: "Jdk" in white on LocalLM Lab's blue rounded square (the same design as the +/// LocalLM Lab app icon). Drawn in code so `swift run` gets it too, with no bundle to hold an image. +enum AppIcon { + static let blue = NSColor(srgbRed: 0x2f / 255, green: 0x6f / 255, blue: 0xed / 255, alpha: 1) + + static func image(size: CGFloat = 512) -> NSImage { + NSImage(size: NSSize(width: size, height: size), flipped: false) { rect in + // Full-bleed rounded square, corner radius as in LocalLM Lab's icon (14 of 64). + blue.setFill() + NSBezierPath(roundedRect: rect, xRadius: size * 14 / 64, yRadius: size * 14 / 64).fill() + + let text = NSMutableAttributedString(string: "J", attributes: [ + .font: NSFont.systemFont(ofSize: size * 0.50, weight: .bold), + .foregroundColor: NSColor.white, + ]) + text.append(NSAttributedString(string: "dk", attributes: [ + .font: NSFont.systemFont(ofSize: size * 0.30, weight: .semibold), + .foregroundColor: NSColor.white, + .kern: -size * 0.004, + ])) + // Centre on the glyphs' ink, not the line box, so the mark sits optically centred. + let ink = text.boundingRect(with: rect.size, options: [.usesLineFragmentOrigin, .usesDeviceMetrics]) + text.draw(at: NSPoint(x: rect.midX - ink.midX, y: rect.midY - ink.midY)) + return true + } + } +} From 28eeb9e2d5196b7b6cb59c21042581ef423edac8 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:34:28 -0700 Subject: [PATCH 33/73] JevDK icon: a small "ev" under the J "Jdk" with "ev" under the J, so the icon also reads "Jev". Checked at Dock size. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/Sources/JevDK/JevDKApp.swift | 36 +++++++++++++-------- 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index d58d2ec..46132b4 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -352,7 +352,8 @@ struct BackendsSheet: View { } /// JevDK's Dock icon: "Jdk" in white on LocalLM Lab's blue rounded square (the same design as the -/// LocalLM Lab app icon). Drawn in code so `swift run` gets it too, with no bundle to hold an image. +/// LocalLM Lab app icon), with a small "ev" under the J for "Jev". Drawn in code so `swift run` +/// gets it too, with no bundle to hold an image. enum AppIcon { static let blue = NSColor(srgbRed: 0x2f / 255, green: 0x6f / 255, blue: 0xed / 255, alpha: 1) @@ -362,18 +363,27 @@ enum AppIcon { blue.setFill() NSBezierPath(roundedRect: rect, xRadius: size * 14 / 64, yRadius: size * 14 / 64).fill() - let text = NSMutableAttributedString(string: "J", attributes: [ - .font: NSFont.systemFont(ofSize: size * 0.50, weight: .bold), - .foregroundColor: NSColor.white, - ]) - text.append(NSAttributedString(string: "dk", attributes: [ - .font: NSFont.systemFont(ofSize: size * 0.30, weight: .semibold), - .foregroundColor: NSColor.white, - .kern: -size * 0.004, - ])) - // Centre on the glyphs' ink, not the line box, so the mark sits optically centred. - let ink = text.boundingRect(with: rect.size, options: [.usesLineFragmentOrigin, .usesDeviceMetrics]) - text.draw(at: NSPoint(x: rect.midX - ink.midX, y: rect.midY - ink.midY)) + func text(_ s: String, _ pt: CGFloat, _ weight: NSFont.Weight, alpha: CGFloat = 1) -> NSAttributedString { + NSAttributedString(string: s, attributes: [ + .font: NSFont.systemFont(ofSize: size * pt, weight: weight), + .foregroundColor: NSColor.white.withAlphaComponent(alpha), + ]) + } + func ink(_ t: NSAttributedString) -> NSRect { + t.boundingRect(with: rect.size, options: [.usesLineFragmentOrigin, .usesDeviceMetrics]) + } + let j = text("J", 0.50, .bold), dk = text("dk", 0.30, .semibold), ev = text("ev", 0.19, .semibold, alpha: 0.85) + let jInk = ink(j), dkInk = ink(dk), evInk = ink(ev) + let gapX = size * 0.015, gapY = size * 0.025 + // Line 1: J and dk on one baseline. Line 2: ev centred under the J. + let width = jInk.width + gapX + dkInk.width + let height = jInk.height + gapY + evInk.height + let left = rect.midX - width / 2, top = rect.midY + height / 2 + let baselineJ = top - jInk.maxY // draw origin y so J's ink top = top + j.draw(at: NSPoint(x: left - jInk.minX, y: baselineJ)) + dk.draw(at: NSPoint(x: left + jInk.width + gapX - dkInk.minX, y: baselineJ)) + let evTop = top - jInk.height - gapY + ev.draw(at: NSPoint(x: left + jInk.width / 2 - evInk.midX, y: evTop - evInk.maxY)) return true } } From 53bc6d15d0ea67ff364b6a7228fc14e2936d7a9d Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:43:00 -0700 Subject: [PATCH 34/73] JevDK: a File menu that says which file is which MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four kinds of file, now grouped and named in the File menu: - Workspace: Open…, Save Workspace, Save Workspace As… (JevDK's own file, saved as .jevdk.json) - For your app: Export Questions for App… (the SDK's DecisionQuestionSet, .decisions.json) - Test data: Import / Export Answers (CSV)…, Append Run to Results CSV… (also on the Batch toolbar) Open now also reads an exported .decisions.json (questions, system instructions, calibration), so a shipped file can come back into JevDK. README gains a Files section; the guide, SDK guide and design docs use the new names. Co-Authored-By: Claude Opus 5.5 --- docs/sdk-guide.md | 4 +-- examples/jevdk/GUIDE.md | 22 +++++++------ examples/jevdk/README.md | 32 +++++++++++++++---- examples/jevdk/Sources/JevDK/AppModel.swift | 24 +++++++++++++- .../jevdk/Sources/JevDK/Calibration.swift | 5 +-- .../Sources/JevDK/CalibrationSheet.swift | 2 +- examples/jevdk/Sources/JevDK/JevDKApp.swift | 26 ++++++++++----- .../jevdk/Sources/JevDK/ResultsView.swift | 6 ++-- .../Sources/OpenJevKit/EditorModel.swift | 16 ++++++++++ 9 files changed, 105 insertions(+), 32 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 8dd68b1..af62bc2 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1911,7 +1911,7 @@ this need the web, which tool, is it sensitive) and a chat model for the text, b Questions are the work, so the SDK carries them from testing to shipping intact: ```swift -// The questions exactly as tested in JevDK (File → Export for App…), with the model, revision, +// The questions exactly as tested in JevDK (File → Export Questions for App…), with the model, revision, // wrapper and calibration they were tested with. let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "support.decisions", withExtension: "json")!) let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // the tested wrapper + calibration @@ -3395,7 +3395,7 @@ struct DecisionCalibration: Codable { func measure(_ samples: [Sample]) -> Measurement func apply(_ probabilities: [Double], kind: Sample.Kind) -> [Double] } -struct DecisionQuestionSet: Codable { // JevDK's "Export for App…" +struct DecisionQuestionSet: Codable { // JevDK's "Export Questions for App…" static let currentFormat: Int var format: Int; var name: String; var questions: [DecisionQuestion]; var tuning: Tuning? struct Tuning { var model: ModelID?; var revision, system, inputLabel: String?; var calibration: DecisionCalibration?; var testedAt: Date? } diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index 8d86bd8..a19b9f0 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -148,7 +148,7 @@ real messages are messier, and the mess is where models go wrong. Include: Paste them into **Batch**, one per line, and **Ask all**. If you already have them in a spreadsheet, save it as CSV with an `input` column and one column per question holding the -correct answer (blank where you haven't decided), and use **Answer set → Import** instead: the +correct answer (blank where you haven't decided), and use **File → Import Answers (CSV)…** instead: the inputs and your marks load together. Yes/no answers can be Yes/No, true/false or 1/0; a choice is its key; a score is its level number or text. Then, under each cell, use **mark…** to record the correct answer. This is the slow part, and the most valuable: from now on every @@ -159,9 +159,9 @@ Tips: - Mark answers as **you** want your app to behave, not as you guess the model will. - If you hesitate over a mark, the question is probably ambiguous. Note it; you'll reword it in step 5. -- Save the set (⌘S), and **Answer set → Export** the inputs and marks as CSV. That CSV is your - test suite: keep it with your app and grow it when you find new failure cases. JevDK also - remembers your workspace between launches. +- **Save Workspace** (⌘S) keeps everything for JevDK. **File → Export Answers (CSV)…** saves the + inputs and marks as a CSV: that's your test suite. Keep it with your app and grow it when you + find new failure cases. JevDK also remembers your workspace between launches. ### Step 4. Run the batch and read the grid @@ -200,7 +200,7 @@ set can tell you. JevDK doesn't keep a leaderboard on purpose. mixture-of-experts models are flagged because their probabilities were unstable in testing. 2. **Backends** (toolbar): optionally add a hosted decider (Featherless, or TypeSafe on OpenRouter) as a reference point. -3. Run the batch with model A, then **Results CSV** → append. Switch to model B, run, append. +3. Run the batch with model A, then **Results CSV** (Batch toolbar) → append. Switch to model B, run, append. 4. Open the CSV in Numbers or Excel. A pivot table of `correct` by `model` and `question` is per-question accuracy; `question_ms` and `decision_ms` are speed. @@ -229,7 +229,7 @@ only use the top answer. and gets sharpened. 3. Read the before → after table. Lower is better on both measures (see section 6). 4. Turn on **Show local results calibrated** to see what your app will see. -5. **Export for App…** (step 8) carries it into your app with the questions. If you change the +5. **Export Questions for App…** (step 8) carries it into your app with the questions. If you change the model or the system instructions later, JevDK flags the calibration until you redo it. What calibration can't do: fix wrong answers. In our test, a confidently wrong answer stayed @@ -238,8 +238,8 @@ wrong; it just became less confidently wrong. Get the questions right first (ste ### Step 8. Ship it Use the same questions, model, system instructions and calibration you tested. **File → -Export for App…** writes them into one file (`support.decisions.json`); add it to your app and -load it: +Export Questions for App…** writes them into one file (`support.decisions.json`); add it to your +app and load it: ```swift let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "support.decisions", withExtension: "json")!) @@ -344,7 +344,11 @@ Sixteen messages are enough to show the method, not to choose a model. Use 50 or and a probability, instead of writing text. - **Noul, Choice, Score:** the three question types: yes/no, one of several options, a point on a scale. (Names from TypeSafe's Jev.) -- **Answer set:** example inputs with the correct answers marked. Your test suite. +- **Answer set:** example inputs with the correct answers marked. Your test suite. JevDK + imports and exports it as CSV (**File → Import / Export Answers**). +- **Workspace:** JevDK's own file (⌘S / ⌘O) with everything you're working on. Not for your app; + for that, **Export Questions for App**. The README's [Files](README.md#files) section compares + the four kinds of file. - **Backend:** where the decider runs: locally (OpenJev on this Mac) or hosted (Featherless, TypeSafe). - **System instructions / wrapper:** the text that frames every question for a local model. diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index d2aa687..a5dbe10 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -70,12 +70,31 @@ again. **Examples** asks before replacing work you've changed, and can save it t Your own questions: **Add question** on the left. Pick **Noul** (yes / no), **Choice** (one of several options) or **Score** (a scale). Give it a short name (what your app reads), the question text (what the model reads), and for Choice / Score the options. **Duplicate** a question -to try a second wording side by side. **⌘S / ⌘O** saves and opens a question set with its inputs. +to try a second wording side by side. **⌘S** saves the whole workspace to a file, **⌘O** opens one +(see [Files](#files)). + +## Files + +JevDK works with four kinds of file. The File menu groups them: + +| File menu | The file | What's in it | Use it to | +|---|---|---|---| +| **Workspace:** Open…, Save Workspace (⌘S), Save Workspace As… | `Customer support.jevdk.json`, JevDK's own format | Everything: questions, system instructions, the input and batch list, your marked answers, the calibration | Pick your work back up, or hand it to a colleague. JevDK also keeps the last workspace between launches on its own. | +| **For your app:** Export Questions for App… (⌘E) | `customer-support.decisions.json`, the SDK's `DecisionQuestionSet` | The questions exactly as tested, plus the model, version, system instructions and calibration they were tested with. No inputs or marks. | Bundle it in your app; `OpenJevDecisionProvider(mlx:tunedWith:)` loads it. | +| **Test data:** Import Answers (CSV)…, Export Answers (CSV)… | `customer-support.answers.csv` | Your batch inputs and the correct answers you marked: an `input` column, then one column per question | Edit test cases in a spreadsheet, share them, and run them in your app's tests with `lab.evaluate`. | +| **Test data:** Append Run to Results CSV… | `jevdk-results.csv`, grows with every run | Every answer from every backend in a run, with the model, timing and whether it was right | Compare models and wordings over time (below). | + +Which one when: while you're working, **Save Workspace**. When the questions are ready to ship, +**Export Questions for App**, and **Export Answers** for your app's tests. **Open…** reads a +workspace or an exported `.decisions.json`, so a file you shipped can come back into JevDK to work +on (its questions, system instructions and calibration; it has no inputs or marks). + +The Batch toolbar has the same answer and results actions (**Answers**, **Results CSV**). ## Comparing models: the results CSV JevDK doesn't tell you which model to use; your questions and your answers do. After a batch, -**Results CSV** appends the run to a CSV file: one row per input × question × backend. Run the +**Results CSV** (Batch toolbar, or File → Append Run to Results CSV…) appends the run to a CSV file: one row per input × question × backend. Run the same batch with another model (or quantization, wrapper or question wording) and append again; the runs line up in one spreadsheet. The first click asks for a file; later clicks append to it (**Append to another CSV…** to switch). A file with a different header is never touched. @@ -106,17 +125,18 @@ for your app: 3. **Calibrate…** → **Fit calibration**. It shows the calibration error and log-loss before → after for each kind of question (aim for 30+ marks per kind). 4. Turn on **Show local results calibrated** to see what your app will see. It's saved with - the question set (⌘S). If you change the model or the system instructions, JevDK flags the + the workspace (⌘S). If you change the model or the system instructions, JevDK flags the calibration until you refit. ## Taking it to your app -- **File → Export for App…** writes a `DecisionQuestionSet` JSON file: the questions plus the +- **File → Export Questions for App…** writes a `DecisionQuestionSet` JSON file: the questions plus the model, revision, wrapper and calibration you tested. Bundle it and load it with `DecisionQuestionSet(contentsOf:)`; `OpenJevDecisionProvider(mlx:tunedWith:)` applies the wrapper and calibration. -- **Answer set → Export** (Batch toolbar, or the File menu) saves your inputs and marks as CSV: - an `input` column and one column per question. **Import** loads one, e.g. from a spreadsheet. +- **File → Export Answers (CSV)…** (also under **Answers** on the Batch toolbar) saves your + inputs and marks as CSV: an `input` column and one column per question. **Import Answers + (CSV)…** loads one, e.g. from a spreadsheet. In your app's tests, `lab.evaluate(route:questions:answerSet:)` scores a backend against it. See [GUIDE.md](GUIDE.md) step 8. diff --git a/examples/jevdk/Sources/JevDK/AppModel.swift b/examples/jevdk/Sources/JevDK/AppModel.swift index 157ba9f..e781c95 100644 --- a/examples/jevdk/Sources/JevDK/AppModel.swift +++ b/examples/jevdk/Sources/JevDK/AppModel.swift @@ -689,7 +689,7 @@ final class AppModel { if url == nil || `as` { let panel = NSSavePanel() panel.allowedContentTypes = [.json] - panel.nameFieldStringValue = set.name.isEmpty ? "questions.json" : "\(set.name).json" + panel.nameFieldStringValue = set.name.isEmpty ? "workspace.jevdk.json" : "\(set.name).jevdk.json" guard panel.runModal() == .OK, let picked = panel.url else { return false } url = picked } @@ -706,9 +706,12 @@ final class AppModel { } } + /// Open a workspace file (what Save Workspace writes), or a questions file exported for an app + /// (`.decisions.json`): its questions and system instructions, and the calibration it carries. func open() { let panel = NSOpenPanel() panel.allowedContentTypes = [.json] + panel.message = "A JevDK workspace, or a questions file exported for an app (.decisions.json)." guard panel.runModal() == .OK, let url = panel.url else { return } do { let data = try Data(contentsOf: url) @@ -719,6 +722,25 @@ final class AppModel { labels = saved.labels ?? [:] calibration = saved.calibration applyCalibration = saved.calibration != nil + } else if let exported = try? DecisionQuestionSet(json: data) { + let t = exported.tuning + set = QuestionSet(name: exported.name, system: t?.system ?? QuestionSet.defaultSystem, + inputLabel: t?.inputLabel ?? QuestionSet.defaultInputLabel, + questions: exported.questions.map(EditableQuestion.init)) + labels = [:] + if let cal = t?.calibration, let model = t?.model { + calibration = FittedCalibration( + noulTemperature: cal.noulTemperature, choiceTemperature: cal.choiceTemperature, + scoreTemperature: cal.scoreTemperature, repoID: model.rest, revision: t?.revision ?? "?", + system: set.system, inputLabel: set.inputLabel, fittedAt: t?.testedAt ?? Date(), samples: [:]) + } else { + calibration = nil + } + applyCalibration = calibration != nil + // An app file isn't a workspace: ⌘S asks where to save one rather than overwriting it. + results = []; batchRows = []; samples = [] + fileURL = nil + return } else { set = try JSONDecoder().decode(QuestionSet.self, from: data) } diff --git a/examples/jevdk/Sources/JevDK/Calibration.swift b/examples/jevdk/Sources/JevDK/Calibration.swift index 45ca080..294eb95 100644 --- a/examples/jevdk/Sources/JevDK/Calibration.swift +++ b/examples/jevdk/Sources/JevDK/Calibration.swift @@ -33,10 +33,10 @@ struct FittedCalibration: Codable, Hashable { } /// What the developer pastes into their app. The questions, wrapper and this calibration travel - /// in the exported question set (File → Export for App…), so the code only loads it. + /// in the exported question set (File → Export Questions for App…), so the code only loads it. var swiftSnippet: String { """ - // Export the question set from JevDK (File → Export for App…) and bundle it with your app. + // Export the questions from JevDK (File → Export Questions for App…) and bundle the file with your app. // It holds the questions, the wrapper and this calibration, fitted on \(samples.values.reduce(0, +)) marked answers // for \(repoID) @ \(revision.prefix(12)) (\(fittedAt.formatted(date: .abbreviated, time: .omitted))). let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "questions.decisions", withExtension: "json")!) @@ -83,6 +83,7 @@ extension DecisionCalibration.Sample.Kind { case .noul: "Noul (yes / no)" case .choice: "Choice" case .score: "Score" + @unknown default: "Other" } } } diff --git a/examples/jevdk/Sources/JevDK/CalibrationSheet.swift b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift index 9593568..6f6b332 100644 --- a/examples/jevdk/Sources/JevDK/CalibrationSheet.swift +++ b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift @@ -124,7 +124,7 @@ struct CalibrationSheet: View { .frame(maxWidth: .infinity, alignment: .leading) .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 6)) HStack { - Button("Export for App…") { model.exportForApp() } + Button("Export Questions for App…") { model.exportForApp() } Button("Copy code") { NSPasteboard.general.clearContents() NSPasteboard.general.setString(c.swiftSnippet, forType: .string) diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index 46132b4..7b69826 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -43,17 +43,27 @@ struct JevDKApp: App { .frame(minWidth: 1060, minHeight: 660) } .commands { + // Four kinds of file (README "Files"): the workspace (JevDK's own, everything), the + // questions for your app (the SDK's DecisionQuestionSet), answers (CSV), and results (CSV). CommandGroup(replacing: .newItem) { - Button("Open Question Set…") { model.open() }.keyboardShortcut("o") + Section("Workspace") { + Button("Open…") { model.open() }.keyboardShortcut("o") + } } CommandGroup(replacing: .saveItem) { - Button("Save") { model.save() }.keyboardShortcut("s") - Button("Save As…") { model.save(as: true) }.keyboardShortcut("s", modifiers: [.command, .shift]) - Divider() - Button("Export for App…") { model.exportForApp() }.keyboardShortcut("e") - Divider() - Button("Import Answer Set…") { model.importAnswerSet() } - Button("Export Answer Set…") { model.exportAnswerSet() } + Section("Workspace") { + Button("Save Workspace") { model.save() }.keyboardShortcut("s") + Button("Save Workspace As…") { model.save(as: true) }.keyboardShortcut("s", modifiers: [.command, .shift]) + } + Section("For your app") { + Button("Export Questions for App…") { model.exportForApp() }.keyboardShortcut("e") + } + Section("Test data") { + Button("Import Answers (CSV)…") { model.importAnswerSet() } + Button("Export Answers (CSV)…") { model.exportAnswerSet() } + Button("Append Run to Results CSV…") { model.appendResults(choose: model.resultsCSVPath == nil) } + .disabled(model.batchRows.isEmpty || model.batchProgress != nil) + } } } } diff --git a/examples/jevdk/Sources/JevDK/ResultsView.swift b/examples/jevdk/Sources/JevDK/ResultsView.swift index 627a22d..f76176c 100644 --- a/examples/jevdk/Sources/JevDK/ResultsView.swift +++ b/examples/jevdk/Sources/JevDK/ResultsView.swift @@ -133,10 +133,10 @@ struct ResultsView: View { Label("calibrated", systemImage: "dial.medium").font(.jCaption).foregroundStyle(.tint) } Menu { - Button("Import answer set (CSV)…") { model.importAnswerSet() } - Button("Export answer set (CSV)…") { model.exportAnswerSet() } + Button("Import Answers (CSV)…") { model.importAnswerSet() } + Button("Export Answers (CSV)…") { model.exportAnswerSet() } } label: { - Label("Answer set", systemImage: "checklist") + Label("Answers", systemImage: "checklist") } .fixedSize() .help("Load inputs with their correct answers from a CSV (input column, then one column per question), or save yours") diff --git a/examples/jevdk/Sources/OpenJevKit/EditorModel.swift b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift index bdd20fb..4621513 100644 --- a/examples/jevdk/Sources/OpenJevKit/EditorModel.swift +++ b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift @@ -79,6 +79,22 @@ public struct EditableQuestion: Codable, Hashable, Identifiable, Sendable { options = try c.decodeIfPresent([AnswerOption].self, forKey: .options) ?? [] } + /// The editor question for an SDK one, e.g. from a `DecisionQuestionSet` exported for an app. + public init(_ q: DecisionQuestion) { + switch q.kind { + case .noul: + self.init(name: q.id, kind: .noul, text: q.instructions) + case .choice(let criteria): + self.init(name: q.id, kind: .choice, text: q.instructions, + options: criteria.map { AnswerOption(key: $0.key, description: $0.description ?? "") }) + case .score(let levels): + self.init(name: q.id, kind: .score, text: q.instructions, + options: levels.enumerated().map { AnswerOption(key: String($0.offset), description: $0.element) }) + @unknown default: + self.init(name: q.id, kind: .noul, text: q.instructions) + } + } + /// The SDK question this becomes. public var sdkQuestion: DecisionQuestion { switch kind { From 65db74de32ba08353c3d3c141305df40249d3503 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:45:40 -0700 Subject: [PATCH 35/73] jevdk: README links the web version of the guide Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index a5dbe10..3160eac 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -23,7 +23,7 @@ Hugging Face cache and Keychain keys. Generated from [`project.yml`](project.yml **A signed, notarized DMG to share:** `VERSION=x.y.z APP_IDENTITY=… KEYCHAIN_PROFILE=… scripts/release_macos.sh` (Developer ID identity, `notarytool` profile; options in the script's header) → `dist/JevDK-x.y.z-arm64.dmg`. -**New to decision models?** Read [the developer's guide](GUIDE.md) first: what a decider is, +**New to decision models?** Read [the developer's guide](GUIDE.md) first ([on the web](https://thisbrain.ai/locallm/jdk-guide.html), with screenshots): what a decider is, when to use one, and the workflow from writing questions to shipping them, with a worked example. ## The idea From fd801f0b7bf21b10a4996865469301d44b6cf5c9 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:48:36 -0700 Subject: [PATCH 36/73] =?UTF-8?q?JevDK:=20"Export=20Questions=E2=80=A6"=20?= =?UTF-8?q?(was=20"Export=20Questions=20for=20App=E2=80=A6")?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The File menu's "For your app" heading already says where it goes. Co-Authored-By: Claude Opus 5.5 --- docs/sdk-guide.md | 4 ++-- examples/jevdk/GUIDE.md | 6 +++--- examples/jevdk/README.md | 6 +++--- examples/jevdk/Sources/JevDK/Calibration.swift | 4 ++-- examples/jevdk/Sources/JevDK/CalibrationSheet.swift | 2 +- examples/jevdk/Sources/JevDK/JevDKApp.swift | 2 +- 6 files changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index af62bc2..7266111 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1911,7 +1911,7 @@ this need the web, which tool, is it sensitive) and a chat model for the text, b Questions are the work, so the SDK carries them from testing to shipping intact: ```swift -// The questions exactly as tested in JevDK (File → Export Questions for App…), with the model, revision, +// The questions exactly as tested in JevDK (File → Export Questions…), with the model, revision, // wrapper and calibration they were tested with. let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "support.decisions", withExtension: "json")!) let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // the tested wrapper + calibration @@ -3395,7 +3395,7 @@ struct DecisionCalibration: Codable { func measure(_ samples: [Sample]) -> Measurement func apply(_ probabilities: [Double], kind: Sample.Kind) -> [Double] } -struct DecisionQuestionSet: Codable { // JevDK's "Export Questions for App…" +struct DecisionQuestionSet: Codable { // JevDK's "Export Questions…" static let currentFormat: Int var format: Int; var name: String; var questions: [DecisionQuestion]; var tuning: Tuning? struct Tuning { var model: ModelID?; var revision, system, inputLabel: String?; var calibration: DecisionCalibration?; var testedAt: Date? } diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index a19b9f0..d781227 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -229,7 +229,7 @@ only use the top answer. and gets sharpened. 3. Read the before → after table. Lower is better on both measures (see section 6). 4. Turn on **Show local results calibrated** to see what your app will see. -5. **Export Questions for App…** (step 8) carries it into your app with the questions. If you change the +5. **Export Questions…** (step 8) carries it into your app with the questions. If you change the model or the system instructions later, JevDK flags the calibration until you redo it. What calibration can't do: fix wrong answers. In our test, a confidently wrong answer stayed @@ -238,7 +238,7 @@ wrong; it just became less confidently wrong. Get the questions right first (ste ### Step 8. Ship it Use the same questions, model, system instructions and calibration you tested. **File → -Export Questions for App…** writes them into one file (`support.decisions.json`); add it to your +Export Questions…** writes them into one file (`support.decisions.json`); add it to your app and load it: ```swift @@ -347,7 +347,7 @@ Sixteen messages are enough to show the method, not to choose a model. Use 50 or - **Answer set:** example inputs with the correct answers marked. Your test suite. JevDK imports and exports it as CSV (**File → Import / Export Answers**). - **Workspace:** JevDK's own file (⌘S / ⌘O) with everything you're working on. Not for your app; - for that, **Export Questions for App**. The README's [Files](README.md#files) section compares + for that, **Export Questions**. The README's [Files](README.md#files) section compares the four kinds of file. - **Backend:** where the decider runs: locally (OpenJev on this Mac) or hosted (Featherless, TypeSafe). diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index 3160eac..c1e1cc0 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -80,12 +80,12 @@ JevDK works with four kinds of file. The File menu groups them: | File menu | The file | What's in it | Use it to | |---|---|---|---| | **Workspace:** Open…, Save Workspace (⌘S), Save Workspace As… | `Customer support.jevdk.json`, JevDK's own format | Everything: questions, system instructions, the input and batch list, your marked answers, the calibration | Pick your work back up, or hand it to a colleague. JevDK also keeps the last workspace between launches on its own. | -| **For your app:** Export Questions for App… (⌘E) | `customer-support.decisions.json`, the SDK's `DecisionQuestionSet` | The questions exactly as tested, plus the model, version, system instructions and calibration they were tested with. No inputs or marks. | Bundle it in your app; `OpenJevDecisionProvider(mlx:tunedWith:)` loads it. | +| **For your app:** Export Questions… (⌘E) | `customer-support.decisions.json`, the SDK's `DecisionQuestionSet` | The questions exactly as tested, plus the model, version, system instructions and calibration they were tested with. No inputs or marks. | Bundle it in your app; `OpenJevDecisionProvider(mlx:tunedWith:)` loads it. | | **Test data:** Import Answers (CSV)…, Export Answers (CSV)… | `customer-support.answers.csv` | Your batch inputs and the correct answers you marked: an `input` column, then one column per question | Edit test cases in a spreadsheet, share them, and run them in your app's tests with `lab.evaluate`. | | **Test data:** Append Run to Results CSV… | `jevdk-results.csv`, grows with every run | Every answer from every backend in a run, with the model, timing and whether it was right | Compare models and wordings over time (below). | Which one when: while you're working, **Save Workspace**. When the questions are ready to ship, -**Export Questions for App**, and **Export Answers** for your app's tests. **Open…** reads a +**Export Questions**, and **Export Answers** for your app's tests. **Open…** reads a workspace or an exported `.decisions.json`, so a file you shipped can come back into JevDK to work on (its questions, system instructions and calibration; it has no inputs or marks). @@ -130,7 +130,7 @@ for your app: ## Taking it to your app -- **File → Export Questions for App…** writes a `DecisionQuestionSet` JSON file: the questions plus the +- **File → Export Questions…** writes a `DecisionQuestionSet` JSON file: the questions plus the model, revision, wrapper and calibration you tested. Bundle it and load it with `DecisionQuestionSet(contentsOf:)`; `OpenJevDecisionProvider(mlx:tunedWith:)` applies the wrapper and calibration. diff --git a/examples/jevdk/Sources/JevDK/Calibration.swift b/examples/jevdk/Sources/JevDK/Calibration.swift index 294eb95..578621d 100644 --- a/examples/jevdk/Sources/JevDK/Calibration.swift +++ b/examples/jevdk/Sources/JevDK/Calibration.swift @@ -33,10 +33,10 @@ struct FittedCalibration: Codable, Hashable { } /// What the developer pastes into their app. The questions, wrapper and this calibration travel - /// in the exported question set (File → Export Questions for App…), so the code only loads it. + /// in the exported question set (File → Export Questions…), so the code only loads it. var swiftSnippet: String { """ - // Export the questions from JevDK (File → Export Questions for App…) and bundle the file with your app. + // Export the questions from JevDK (File → Export Questions…) and bundle the file with your app. // It holds the questions, the wrapper and this calibration, fitted on \(samples.values.reduce(0, +)) marked answers // for \(repoID) @ \(revision.prefix(12)) (\(fittedAt.formatted(date: .abbreviated, time: .omitted))). let set = try DecisionQuestionSet(contentsOf: Bundle.main.url(forResource: "questions.decisions", withExtension: "json")!) diff --git a/examples/jevdk/Sources/JevDK/CalibrationSheet.swift b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift index 6f6b332..652da0e 100644 --- a/examples/jevdk/Sources/JevDK/CalibrationSheet.swift +++ b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift @@ -124,7 +124,7 @@ struct CalibrationSheet: View { .frame(maxWidth: .infinity, alignment: .leading) .background(.quaternary.opacity(0.5), in: RoundedRectangle(cornerRadius: 6)) HStack { - Button("Export Questions for App…") { model.exportForApp() } + Button("Export Questions…") { model.exportForApp() } Button("Copy code") { NSPasteboard.general.clearContents() NSPasteboard.general.setString(c.swiftSnippet, forType: .string) diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index 7b69826..c6a927f 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -56,7 +56,7 @@ struct JevDKApp: App { Button("Save Workspace As…") { model.save(as: true) }.keyboardShortcut("s", modifiers: [.command, .shift]) } Section("For your app") { - Button("Export Questions for App…") { model.exportForApp() }.keyboardShortcut("e") + Button("Export Questions…") { model.exportForApp() }.keyboardShortcut("e") } Section("Test data") { Button("Import Answers (CSV)…") { model.importAnswerSet() } From fa739daba85a1b8079f1c3cf6a7bbb23a22c26ba Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:54:52 -0700 Subject: [PATCH 37/73] JevDK: About panel with the icon, version and a link to the decision-models page MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit JevDK → About shows the Jdk icon, the app's version ("development build" under swift run, which has no bundle), the SDK version, and a link to thisbrain.ai/locallm/jev.html. The Xcode build's Info.plist now takes its version from project.yml (MARKETING_VERSION / CURRENT_PROJECT_VERSION) instead of a hard-coded 1.0; the release script still sets it for a DMG. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/Sources/JevDK/JevDKApp.swift | 37 +++++++++++++++++++++ examples/jevdk/xcodeproj/JevDK-Info.plist | 4 +-- 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index c6a927f..0e9287e 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -45,6 +45,9 @@ struct JevDKApp: App { .commands { // Four kinds of file (README "Files"): the workspace (JevDK's own, everything), the // questions for your app (the SDK's DecisionQuestionSet), answers (CSV), and results (CSV). + CommandGroup(replacing: .appInfo) { + Button("About JevDK") { AboutPanel.show() } + } CommandGroup(replacing: .newItem) { Section("Workspace") { Button("Open…") { model.open() }.keyboardShortcut("o") @@ -361,6 +364,40 @@ struct BackendsSheet: View { } } +/// JevDK → About: the icon, the version, and where to read about decision models. +enum AboutPanel { + static let webPage = URL(string: "https://thisbrain.ai/locallm/jev.html")! + + /// The app's version when it runs as a built app (Xcode, or the DMG's), else "development build" + /// (`swift run` has no bundle to carry one). + static var version: String { + let info = Bundle.main.infoDictionary + guard let short = info?["CFBundleShortVersionString"] as? String else { return "development build" } + let build = info?["CFBundleVersion"] as? String + return build.map { $0 == short ? short : "\(short) (\($0))" } ?? short + } + + static func show() { + let body = NSFont.systemFont(ofSize: NSFont.smallSystemFontSize) + let centered = NSMutableParagraphStyle() + centered.alignment = .center + let credits = NSMutableAttributedString( + string: "A playground for decision-model (Jev) questions.\nBuilt on the LocalLM Lab SDK \(LocalLMLabSDKVersion.current).\n\n", + attributes: [.font: body, .foregroundColor: NSColor.secondaryLabelColor, .paragraphStyle: centered]) + credits.append(NSAttributedString(string: "Decision models (Jev) in your app", attributes: [ + .font: body, .link: webPage, .paragraphStyle: centered, + ])) + NSApplication.shared.orderFrontStandardAboutPanel(options: [ + .applicationIcon: AppIcon.image(), + .applicationName: "JevDK", + .applicationVersion: version, + .version: "", // the build number is already in applicationVersion + .credits: credits, + ]) + NSApplication.shared.activate() + } +} + /// JevDK's Dock icon: "Jdk" in white on LocalLM Lab's blue rounded square (the same design as the /// LocalLM Lab app icon), with a small "ev" under the J for "Jev". Drawn in code so `swift run` /// gets it too, with no bundle to hold an image. diff --git a/examples/jevdk/xcodeproj/JevDK-Info.plist b/examples/jevdk/xcodeproj/JevDK-Info.plist index cf1d69e..13c740a 100644 --- a/examples/jevdk/xcodeproj/JevDK-Info.plist +++ b/examples/jevdk/xcodeproj/JevDK-Info.plist @@ -17,9 +17,9 @@ CFBundlePackageType APPL CFBundleShortVersionString - 1.0 + $(MARKETING_VERSION) CFBundleVersion - 1 + $(CURRENT_PROJECT_VERSION) LSApplicationCategoryType public.app-category.developer-tools LSMinimumSystemVersion From b272fdf37e0bc2096b88f52fc8e930381c021e7a Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 09:59:40 -0700 Subject: [PATCH 38/73] jevdk release script: duplicate keychain entries of one certificate aren't ambiguous The same Developer ID certificate is often listed several times by find-identity (login, System and iCloud keychains), each with the same SHA-1, and the script refused even the hash as matching "more than one identity". It now counts distinct certificates; codesign accepts the hash. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/scripts/release_macos.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/examples/jevdk/scripts/release_macos.sh b/examples/jevdk/scripts/release_macos.sh index 9f568d0..e00921b 100755 --- a/examples/jevdk/scripts/release_macos.sh +++ b/examples/jevdk/scripts/release_macos.sh @@ -115,7 +115,9 @@ if ! security find-identity -v -p codesigning | grep -F "$APP_IDENTITY" >/dev/nu security find-identity -v -p codesigning || true exit 1 fi -if [[ "$(security find-identity -v -p codesigning | grep -cF "$APP_IDENTITY")" -gt 1 ]]; then +# Ambiguous only if the name matches different certificates. The same certificate is often listed +# several times (login, System and iCloud keychains), each with the same SHA-1; that's fine. +if [[ "$(security find-identity -v -p codesigning | grep -F "$APP_IDENTITY" | awk '{print $2}' | sort -u | wc -l | tr -d ' ')" -gt 1 ]]; then echo "APP_IDENTITY matches more than one identity; pass its SHA-1 hash instead: $APP_IDENTITY" >&2 security find-identity -v -p codesigning | grep -F "$APP_IDENTITY" >&2 || true exit 1 From 829a4021a0203b5143ea5a6725fa8d60550a4775 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 10:23:50 -0700 Subject: [PATCH 39/73] 2.0 docs: decision models (Jev) in the CHANGELOG, README, SDK guide status, tested models and SDK skill MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - CHANGELOG 2.0.0: Added — decision models (Core, Inference, Remote), jevdk under examples, and LocalLMLabState.init's defaulted decisionRoutes. - README: decision models under Building on the SDK, jevdk in the examples list, and the migration callout points at migrating-to-2.0.md. - sdk-guide.md status note describes 2.0 (was 1.0.0 GA) and links what's new. - tested-models.md: which MLX models work as a decider (OpenJev), point-in-time 2026-10-03. - skills/locallmlab-swift-app: when to use lab.decide and how, jevdk in the example selector, decision search terms, migrating-to-2.0 and the jev / jdk-guide pages in the doc map. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 43 ++++++++++++++++++- README.md | 18 ++++++-- docs/sdk-guide.md | 13 ++++-- docs/tested-models.md | 20 +++++++++ skills/locallmlab-swift-app/SKILL.md | 5 ++- .../references/doc-map.md | 4 +- .../references/example-selector.md | 1 + 7 files changed, 93 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c08fbd..345546e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,7 +43,8 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) `liveUpdates`, …): a state JSON encoded under 1.0 no longer decodes. Restore servers with `restore(from:)`. - **Defaulted parameters added**: `SessionOptions.init` and `MLXModelProvider.init` gain - `maxModelCallsPerTurn:` (usual call sites compile unchanged). + `maxModelCallsPerTurn:`; `LocalLMLabState.init` gains `decisionRoutes:` (usual call sites + compile unchanged; snapshots saved by 1.0 still decode, with no decision routes). - **One declaration each** where 1.x had kept two (usual call sites compile unchanged): `makeSession` (now with `restoring:` and `mcpAppHints:`), `respond` / `streamResponse` (with `options:` and `fromAppInstance:`), `LocalLMLab.Configuration.init` (with `mcp: MCPSettings`), @@ -153,6 +154,42 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) which model call of the turn, since when — so an app can say why nothing has appeared yet. - **`maxModelCallsPerTurn`** on `MLXModelProvider` and `SessionOptions`: the tool-loop limit, 16 by default. +### Added — decision models (Jev) + +A **decision model** answers a few typed questions about one input — yes / no, one of several +options, a point on a scale — with a probability for every option, and writes no text. One API +asks them of TypeSafe's Jev, Featherless's Simple Jev, or **OpenJev**, the SDK's own decider on a +local MLX model. Introduction: [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html); +API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdecide). + +- **Core:** `LocalLMLab.decide(route:state:questions:)`; questions in Jev's vocabulary + (`DecisionQuestion.noul` / `.choice` / `.score`, `DecisionState`, `DecisionRequest`, + `DecisionLimits` incl. `maxStateCharacters`); answers (`Decision`, `DecisionAnswer`, + `ChoiceAnswer`, `ScoreAnswer`, `DecisionFidelity`, `DecisionUsage`); `DecisionError`; + `DecisionProvider` for custom backends. Decision routes and providers are separate from chat: + `ModelRegistry.register(decision:)`, `replace(decision:)`, `removeDecisionProvider(scheme:)`, + `route(decision:to:)`, `modelID(forDecision:)`, `decisionProvider(for:)`, + `decisionAvailability(for:)`; saved by `lab.snapshot()`. +- **A decider next to your chat model:** `ModelRegistry.pair(decision:generator:)` checks both + models fit the memory budget and keeps them loaded together (or shares one model); + `DecisionPairing`, `PairableDecisionProvider`, `DecisionError.pairing`. +- **From testing to shipping:** `DecisionQuestionSet` (the tested questions as a JSON file, with + the model, revision, wrapper and calibration they were tested with); `DecisionAnswerSet` + (inputs with the correct answers, as CSV); `LocalLMLab.evaluate(route:questions:answerSet:)` + → `DecisionEvaluation` (accuracy per question, misses, confidence when right and wrong) for + regression tests in CI; `DecisionCalibration` (temperature scaling per question kind: `fit`, + `measure`, `apply`). +- **`LocalLMLabSDKInference` (macOS 27):** `OpenJevDecisionProvider` — answers each question in + one forward pass over the allowed labels' probabilities, no generation, the input read once for + all questions (about 150 ms for four questions on Qwen3 4B). `OpenJevWrapper` (+ `.default`), + `calibration:`, `init(mlx:tunedWith:)`, `tuningMismatch(for:)`, `warnings(for:)` + (`OpenJevWarning.mixtureOfExperts`: use a dense model as the decider), + `decideWithDiagnostics(_:using:)` (`OpenJevDiagnostics`: exact prompts, label mass, timing). +- **`LocalLMLabSDKRemote`:** `JevDecisionProvider` with `JevProviderConfig.openRouter(key:)` + (TypeSafe's Jev via OpenRouter), `.featherless(key:)` and `.featherlessDemo()` (no key, + rate-limited); `validated()`, `requestBody(model:request:)`. Off until registered; the + questions and input are then sent to that service. + ### Fixed - **`contextBudget`** reported the whole session's cumulative usage as the last turn's, so `fractionUsed` grew @@ -160,6 +197,10 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) ### Added — examples +- [`jevdk`](examples/jevdk/): JevDK, a playground for decision-model questions — run them on a + local MLX model and on hosted Jev side by side, score a batch against your marked answers, + calibrate, compare models in a results CSV, and export the tested questions for your app. With a + plain-English [developer's guide](examples/jevdk/GUIDE.md) ([on the web](https://thisbrain.ai/locallm/jdk-guide.html)). - [`mcp-chat`](examples/mcp-chat/): a chat with a local model (Qwen3 8B via MLX, or Apple's on-device model) that shows MCP Apps views inline, with streaming, per-server trust and approval, the date every turn, and conversations that reopen. diff --git a/README.md b/README.md index 5a4f801..be25aa5 100644 --- a/README.md +++ b/README.md @@ -91,9 +91,9 @@ what's public here is: > **The model layer builds for macOS 26 & 27.** `SystemModelProvider` works on > macOS 26; MLX, `ClaudeForFoundationModels`, the hosted providers, and Private Cloud Compute -> need macOS 27. Coming from `0.8.x`? See -> **[docs/migrating-to-1.0.md](docs/migrating-to-1.0.md)** — it's mostly additive, with one -> enum-resilience compile caveat. +> need macOS 27. Coming from 1.x? See **[docs/migrating-to-2.0.md](docs/migrating-to-2.0.md)**: +> 2.0 is a one-time breaking release (mostly tool authorization), then source compatible for every +> 2.x. From `0.8.x`, start with [docs/migrating-to-1.0.md](docs/migrating-to-1.0.md). Two ways to turn a connector or MCP server into something the on-device model can actually call as a tool. **Path A**: drop in a ready-made `Tool` Core already ships for it — @@ -105,6 +105,12 @@ tool names, schemas, and descriptions. Neither is the "real" one — both ship i can mix them. See [`docs/sdk-guide.md` §7a](docs/sdk-guide.md#7a-two-paths-to-tool-calling-ready-made-tools-or-write-your-own) for the full framing. +- **Decision models (Jev), new in 2.0** — `lab.decide` asks typed questions (yes / no, one of + several, a scale) and gets one allowed answer each with probabilities, no text, from TypeSafe's + Jev, Featherless or **OpenJev** on a local MLX model; a small decider can stay loaded next to + your chat model. Start with [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html), + then [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdecide) and + [examples/jevdk/](examples/jevdk/). - **[docs/sdk-guide.md](docs/sdk-guide.md)** — the full developer guide: linking Core, the model layer (§6a), entitlements, all three MCP auth types, Keychain storage, App Sandbox/MAS signing, ready-made vs. hand-written tool-calling (§7a), and a full function/type reference (§12). @@ -164,6 +170,12 @@ Roughly simplest to fullest — every one runnable, with full annotated source i - **[examples/mcp-chat/](examples/mcp-chat/)** (2.0) — a chat with a local model where a tool call that has an MCP App shows the server's interactive widget inline (Todoist's task list, a system monitor); the widget's own calls go through the same per-server approval as the model's. +- **[examples/jevdk/](examples/jevdk/)** (2.0) — JevDK, a playground for decision-model (Jev) + questions: run them on a local MLX model and on hosted Jev side by side, score a batch against + your marked answers, calibrate, compare models, and export the tested questions for your app. + [Developer's guide](https://thisbrain.ai/locallm/jdk-guide.html); + [download the app](https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/JevDK-0.1.0-arm64.dmg) + (macOS 27, signed and notarized, a preview built on SDK 2.0.0-dev). Plus **[examples/api-lab/](examples/api-lab/)** (scripts + a chat app for the OpenAI-compatible endpoint) and **[examples/localai-cli/](examples/localai-cli/)** / **[localai-cli-swift/](examples/localai-cli-swift/)** diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 7266111..594f625 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -9,10 +9,15 @@ below for the working reference apps this guide is drawn from. Requires macOS 26+ on Apple Silicon, Swift 6 tools. -**Status note**: `1.0.0` is generally available — source compatible from `1.0.0-RC.1` onward (see -[§9](#9-scope-and-boundaries-what-core-doesnt-do) for the compatibility policy, or -[`migrating-to-1.0.md`](migrating-to-1.0.md) if you're coming from `0.8.x`). This guide describes -the API as of `1.0.0` GA. `Components` in particular is newer and smaller than `Core`. +**Status note**: this guide describes the API as of `2.0.0`, a one-time breaking release; every 2.x +release after it is source compatible with it (see [§9](#9-scope-and-boundaries-what-core-doesnt-do) +for the compatibility policy). Coming from 1.x: [`migrating-to-2.0.md`](migrating-to-2.0.md); from +`0.8.x`: [`migrating-to-1.0.md`](migrating-to-1.0.md) first. New in 2.0: per-server trust and tool +approval ([§7c](#7c-tool-authorization-two-levers--which-tools-and-whether-they-ask-first)), MCP +Apps ([§3f](#3f-mcp-apps-showing-a-servers-interactive-views)), chat-app support +([Building a chat app](#building-a-chat-app--hosttranscript-streamresponse-turncontext)) and +decision models ([§6c](#6c-decision-models-jev--labdecide)). `Components` in particular is newer and +smaller than `Core`. ## Start here: run a real example before reading further diff --git a/docs/tested-models.md b/docs/tested-models.md index a789da2..48de775 100644 --- a/docs/tested-models.md +++ b/docs/tested-models.md @@ -32,6 +32,26 @@ built-in `ClockTool` cross-check. | **Phi-4-mini**, **SmolLM3-3B** | ❌ | Fail on the *input* side: templates try to list tools via a convention `swift-transformers` doesn't supply. Structurally unreachable regardless of the model. SmolLM3 then fabricates badly. | | **gpt-oss-20B** | ⚠️ app-side gap | The model correctly formats a real call — in OpenAI's "Harmony" format, which `mlx-swift-lm` has no parser for, so the call is never recognized. Not a model gap; a known limitation of `mlx-swift-lm`'s parser. | +## As a decider (OpenJev, `lab.decide`) + +Point-in-time too: **2026-10-03**, SDK `2.0.0-dev`, a 64 GB Apple-silicon Mac, the SDK's own +decision-question evaluation and JevDK's customer-support example. A decider answers typed +questions from the probabilities of each option's label, with no generation +([`sdk-guide.md` §6c](sdk-guide.md#6c-decision-models-jev--labdecide)), so what matters is +different from tool calling. **Test your own questions on your own marked inputs in +[JevDK](../examples/jevdk/); these rows only say where to start.** + +| Model | As a decider | Notes | +|---|---|---| +| **Qwen3-4B-4bit** (`mlx-community/Qwen3-4B-4bit`) | ✅ the tested default | About 150 ms for four questions, about 2.3 GB. 89–93% on the evaluation with its best wording per question; 58/64 on JevDK's customer-support example. Says 100% even when wrong, so calibrate before using its confidence, and expect some questions (in our test, "is the customer blocked?") to need rewording or a hosted decider. | +| **Qwen3.8-27B-4bit** | ✅ | 92% on the triage questions, but about 15 GB, and slower as a decider (about 1.2 s per triage case: the SDK's shared cache loads it as a vision model). Mostly worth it as the chat model, with a small decider beside it (`pair(decision:generator:)`). | +| **Qwen3-1.7B-4bit** | ⚠️ too weak | Fast, but 74% on answer checks and about 50% on "does this need live data?". | +| **Mixture-of-experts models** | ❌ not as a decider | Their probabilities moved by up to 0.45 depending on how the prompt was split. `OpenJevDecisionProvider.warnings(for:)` flags them. Use one as the chat model instead. | + +For comparison, the hosted Featherless classifiers (Qwen3.8-27B and Qwen3.5-4B) scored 61/64 on +the same customer-support example, and TypeSafe's Jev is trained to decide and reports calibrated +probabilities. + ## Capacity notes (this Mac, this workload) - **Apple on-device** (`SystemModelProvider` / `.system`): ~8,192-token context window. A diff --git a/skills/locallmlab-swift-app/SKILL.md b/skills/locallmlab-swift-app/SKILL.md index ab6f24b..f639502 100644 --- a/skills/locallmlab-swift-app/SKILL.md +++ b/skills/locallmlab-swift-app/SKILL.md @@ -1,6 +1,6 @@ --- name: locallmlab-swift-app -description: Build, debug, or modify macOS Swift and SwiftUI apps that use the LocalLM Lab SDK, including Core connectors, MCP, Components, model providers, sandboxing, signing, and example-based integration patterns. +description: Build, debug, or modify macOS Swift and SwiftUI apps that use the LocalLM Lab SDK, including Core connectors, MCP, Components, model providers, decision models (Jev, `lab.decide`), sandboxing, signing, and example-based integration patterns. --- # LocalLM Lab Swift App @@ -15,7 +15,7 @@ Before editing SDK integration code, read the nearest maintained local reference - For choosing an example to copy or diff, use [references/example-selector.md](references/example-selector.md). - For app-bundle, TCC, OAuth, sandbox, and signing work, use [references/macos-integration-checklist.md](references/macos-integration-checklist.md). -If the public API is uncertain, check `docs/api-surface.md` rather than guessing. If the task mentions migration from `0.8.x` or a prior beta, read `docs/migrating-to-1.0.md`. +If the public API is uncertain, check `docs/api-surface.md` rather than guessing. If the task mentions migration from 1.x, read `docs/migrating-to-2.0.md`; from `0.8.x` or a 1.0 beta, `docs/migrating-to-1.0.md`. ## Default Decisions @@ -25,6 +25,7 @@ If the public API is uncertain, check `docs/api-surface.md` rather than guessing - Treat Info.plist usage strings, entitlements, OAuth callback routing, App Sandbox settings, and final bundle signing as part of the implementation, not packaging afterthoughts. - For model-layer work, register macOS-27-only providers behind availability checks and link only the SDK modules the app actually uses. - For MLX/open-weight model work, validate and probe model capability instead of assuming a model can tool-call. +- When the app needs a small, fixed decision about some text (route a request, pick a team, spot personal information, check an answer), use a decision model with `lab.decide` (`docs/sdk-guide.md` §6c) instead of asking a chat model and parsing its reply. Write typed questions (`.noul`, `.choice`, `.score`), put any context in the question or the input (there is no system prompt), and only ask what the input and general knowledge can answer, never current facts. Locally, use `OpenJevDecisionProvider` on a small dense MLX model (Qwen3-4B-4bit is the tested default; not a mixture-of-experts model) and `lab.models.pair(decision:generator:)` to keep it loaded next to the chat model. Don't threshold uncalibrated `.tokenScored` confidence. Have the developer test the questions on marked inputs in `examples/jevdk` and ship the exported `DecisionQuestionSet`, with `lab.evaluate` over its answer set as a regression test. ## Verification diff --git a/skills/locallmlab-swift-app/references/doc-map.md b/skills/locallmlab-swift-app/references/doc-map.md index 6e79055..77ab662 100644 --- a/skills/locallmlab-swift-app/references/doc-map.md +++ b/skills/locallmlab-swift-app/references/doc-map.md @@ -7,12 +7,13 @@ Use these local files as the source of truth while working in this repository. - `docs/sdk-guide.md` is the main developer guide. Use it for linking SDK modules, choosing SDK vs toolkit, MCP auth, Keychain storage, connector setup, Path A vs Path B, model providers, Workspace tools, Components, App Sandbox, signing, and the narrative API reference. - `docs/api-surface.md` is the generated public API inventory. Use it to confirm names, signatures, enum cases, and module boundaries before writing code. - `docs/annotated-examples.md` contains full annotated source for every reference app with SDK touchpoints marked inline. Use it when the user wants to understand how much SDK-specific code is needed, or when a small source excerpt is easier than opening an example tree. +- `docs/migrating-to-2.0.md` covers 1.x to 2.0: tool authorization per MCP server, new MCP enum cases, and what's new (chat apps, MCP Apps, decision models). - `docs/migrating-to-1.0.md` covers `0.8.x` to `1.0` migration, beta caveats, non-frozen enum switches requiring `@unknown default`, and module split details. - `docs/tested-models.md` is a point-in-time MLX model snapshot. Treat it as a starting point only; `MLXModelProvider.capabilityProbe` is authoritative for a user's chosen model. ## Public Site -The public site is `https://thisbrain.ai/locallm/`. Prefer local repo docs while editing code. Use the website when the user asks for public-facing wording, external links, or developer-facing copy that should match the published site. +The public site is `https://thisbrain.ai/locallm/`. Prefer local repo docs while editing code. Use the website when the user asks for public-facing wording, external links, or developer-facing copy that should match the published site. For decision models, `jev.html` is the plain-English introduction (with links to TypeSafe's, OpenRouter's and Featherless's own docs) and `jdk-guide.html` is the JevDK workflow. ## Common Local Search Terms @@ -22,3 +23,4 @@ The public site is `https://thisbrain.ai/locallm/`. Prefer local repo docs while - Model layer: `LocalLMLab`, `ModelProvider`, `SystemModelProvider`, `PCCModelProvider`, `MLXModelProvider`, `RemoteModelProvider`, `RouteName`, `LocalLMLabSession`. - Workspace: `WorkspaceAccess`, `WorkspaceTools`, `security-scoped bookmark`, `FileBackedTool`. - Components: `MCPServerPickerView`, `MCPServerManagerObservable`, `ModelPickerView`, `AIModelsSettingsView`. +- Decision models: `lab.decide`, `DecisionQuestion`, `DecisionProvider`, `OpenJevDecisionProvider`, `JevDecisionProvider`, `JevProviderConfig`, `pair(decision:generator:)`, `DecisionQuestionSet`, `DecisionAnswerSet`, `DecisionCalibration`, `evaluate(route:`. diff --git a/skills/locallmlab-swift-app/references/example-selector.md b/skills/locallmlab-swift-app/references/example-selector.md index 9275f33..1b50346 100644 --- a/skills/locallmlab-swift-app/references/example-selector.md +++ b/skills/locallmlab-swift-app/references/example-selector.md @@ -15,6 +15,7 @@ Pick the nearest reference app before inventing a new integration shape. - `examples/components-demo`: prebuilt Components UI for adding and managing MCP servers. - `examples/model-switch`: Core plus Remote plus Components for online providers, API keys, model selection, web search, and citations. - `examples/aiql`: SwiftUI "ask your data" app using MCP, `FileBackedTool`, data verbs, CSV output, sandboxing, and OAuth. +- `examples/jevdk` (2.0): JevDK, a SwiftUI playground for decision models: `lab.decide` across local `OpenJevDecisionProvider` (Inference) and hosted `JevDecisionProvider` (Remote), batch scoring against marked answers, `DecisionCalibration`, and `DecisionQuestionSet` / `DecisionAnswerSet` export. The reference for any decision-model integration; its `GUIDE.md` is the workflow. ## Toolkit Examples From 0e6d8e158232fe7be492d1ca3d0442ab9dc425d0 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 10:41:25 -0700 Subject: [PATCH 40/73] sdk-guide: measuring and guarding a local model's turn (2.0) New subsection after MLXModelProvider: reading a turn's mlx.* usage metadata (MLXUsageMetadataKey table, code), why languageModelSession.usage is the whole session, tool rounds re-reading the conversation, short-prompt pp; MLXPrefillMonitor for "what is it reading"; the up-front refusal (retried by retryOnContextOverflow) and maxModelCallsPerTurn with provider/session examples and precedence. ContextBudget + RetryPolicy: what contextBudget measures now, and that local refusals are retried. Co-Authored-By: Claude Opus 5.5 --- docs/sdk-guide.md | 89 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 87 insertions(+), 2 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 594f625..d22bdd0 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1241,6 +1241,84 @@ If you would rather store it yourself, persist `(repoID, resolvedRevision)` anyw under the Hugging Face cache directory (`remove(_:)` deletes that whole directory, refs and all), and pass it back in `pinnedRevisions` before redownloading. +### Measuring and guarding a local model's turn — usage, speed, memory, limits + +> **Use it when** you want to show how a local model performed on a turn (tokens, prompt and +> generation speed, memory), say why nothing has appeared yet on a long prompt, or decide how many +> tool rounds one turn may take. New in 2.0; MLX models only — other providers don't report these. +> +> **Reference implementation:** LocalLM Lab's Playground **Benchmark** panel reads all of it. + +**Usage and speed, per turn.** Each turn's response carries the model's real token counts and an +`mlx.*` metadata dictionary of turn totals — read it from the response's `usage`, or the last stream +snapshot's: + +```swift +var last: LanguageModelSession.Usage? +for try await snapshot in session.languageModelSession.streamResponse(to: prompt) { last = snapshot.usage } +let meta = last?.metadata ?? [:] +func number(_ key: String) -> Double? { meta[key].flatMap { Double("\($0)") } } + +let tg = number(MLXUsageMetadataKey.tokensPerSecond) // generation, t/s +let pp = number(MLXUsageMetadataKey.promptTokensPerSecond) // prompt processing, t/s +let calls = number(MLXUsageMetadataKey.rounds) // 1 + one per tool round +``` + +| `MLXUsageMetadataKey` | Value (turn total) | +|---|---| +| `promptTokens`, `generatedTokens` | Tokens read and written, summed over the turn's model calls | +| `promptSeconds`, `generateSeconds` | Time in each phase | +| `promptTokensPerSecond` (pp), `tokensPerSecond` (tg) | Σ tokens ÷ Σ seconds — absent if no time was measured | +| `rounds` | Model calls: the first, plus one per round after tool calls | +| `stopReason` | `stop`, `length` or `cancelled` | +| `draftTokensProposed`, `draftTokensAccepted`, `speculativeRounds` | With a speed helper (`pairDraftModel`) only | +| `residentMemoryBytes`, `peakMemoryBytes`, `gpuMemoryLimitBytes` | MLX memory when the turn began (the loaded weights), its peak during the turn, and how much the GPU may use | + +Three things to know when reading usage: + +- **`languageModelSession.usage` is the whole session**, summed over every model call — it grows + each turn. One turn's figures are on that turn's response, as above. +- **A tool round re-reads the whole conversation.** Each model call in a turn processes everything + before it again, so `promptTokens` for a tool turn is roughly *calls × conversation*, and one tool + call can double a long prompt's reading time. Prefer giving the model facts directly over a tool + it must call each turn — e.g. the date through [`turnContext`](#building-a-chat-app--hosttranscript-streamresponse-turncontext), not a clock tool. +- **pp from a short prompt is mostly overhead.** Measure prefill speed on a prompt of a few + thousand tokens. + +**What is it reading right now?** A long prompt can take tens of seconds before the first token. +`MLXPrefillMonitor.active` lists prompts being processed in this process — exact tokens, which model +call of the turn, and since when — from the moment prefill starts until the first output: + +```swift +if let reading = MLXPrefillMonitor.active.last(where: { $0.repoID == repoID }), + Date().timeIntervalSince(reading.startedAt) > 3 { + status = "Reading a \(reading.promptTokens.formatted())-token prompt (model call \(reading.modelCall))" +} +``` + +Poll it while a turn has produced no output (once a second is plenty); it's a cheap read. + +**Guards the SDK applies before every model call.** + +- **A prompt that can't be processed is refused up front** — over the model's context window + (`max_position_embeddings`), or one whose cache wouldn't fit in the GPU memory left — with + `LocalLMLabError.context` and a plain message, instead of starting a prefill that would fail or + crawl. `respond` / `streamResponse` treat it as a context overflow, so + [`retryOnContextOverflow`](#contextbudget--retrypolicy--surviving-a-long-session) compacts and + retries it. +- **A turn that keeps calling tools stops** after `maxModelCallsPerTurn` model calls (default + `MLXModelProvider.defaultMaxModelCallsPerTurn`, 16: the first plus 15 tool rounds), with an error + saying the model kept calling tools without answering. Set your app's default on the provider, or + per session: + +```swift +let mlx = MLXModelProvider(maxModelCallsPerTurn: 40) // this app's default +let agent = try lab.makeSession(route: .local, tools: tools, + options: .init(maxModelCallsPerTurn: .max)) // this session: no limit +``` + + Session beats provider beats 16; values below 1 count as 1. + ### Pinning, updating and cleaning up model versions *Part of [supply-chain hardening](#mlxmodelprovider--run-open-weight-models-locally-locallmlabsdkinference) for model downloads: the trust policy and verification above decide whether to fetch; pins decide which version you keep.* @@ -1608,11 +1686,18 @@ from `respond` / `streamResponse` (it's their retry), and `.serverToolCall` once > each run. - `session.contextBudget` — `windowTokens`, `lastInputTokens`, `fractionUsed` (best-effort). - Show a "context 78% full" gauge; decide when to start a fresh session. + Show a "context 78% full" gauge; decide when to start a fresh session. It reflects the latest + turn's last model call — the conversation as the model last read it — recorded by `respond` / + `streamResponse` (a turn run on `languageModelSession` directly isn't seen). Not + `languageModelSession.usage`, which sums every call in the session. `windowTokens` is a per-model + estimate, `nil` for downloaded models; for those, take the window from the model's + `config.json` (`InstalledModel.contextTokens`). - `session.retryOnContextOverflow = RetryPolicy(maxRetries: 2, compact: { transcript in myTrim(transcript) })` — when a turn throws `contextSizeExceeded`, the SDK calls your `compact` hook, rebuilds the session with the smaller transcript, emits `.contextCompacted`, - and retries. Applies to turns run with `respond` / `streamResponse`; a turn run on + and retries. A local model's up-front refusal of a too-long prompt (`LocalLMLabError.context`, + [above](#measuring-and-guarding-a-local-models-turn--usage-speed-memory-limits)) is retried the + same way. Applies to turns run with `respond` / `streamResponse`; a turn run on `languageModelSession` directly isn't retried ([Running a turn](#running-a-turn-and-when-to-use-languagemodelsession)). With no `compact` hook it just rethrows. From 40ad53c5bbf50d0260f13146e9800b91d43fef4c Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 14:54:49 -0700 Subject: [PATCH 41/73] JevDK: drop internal shorthand and the old-name Keychain migration - The editor tip read "Tips from the eval" (an internal tool); now "Tips". Two comments said "the eval's pick" and "chosen with the eval, D1b" (an internal phase label). - Keychain no longer migrates keys from the app's earlier internal names (AskJev, JevLab): no public user had them, and AskJev is another product's name. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/Sources/JevDK/AppModel.swift | 4 ++-- examples/jevdk/Sources/JevDK/EditorView.swift | 2 +- examples/jevdk/Sources/JevDK/Keychain.swift | 11 +---------- examples/jevdk/Sources/OpenJevKit/EditorModel.swift | 2 +- 4 files changed, 5 insertions(+), 14 deletions(-) diff --git a/examples/jevdk/Sources/JevDK/AppModel.swift b/examples/jevdk/Sources/JevDK/AppModel.swift index e781c95..5736ce9 100644 --- a/examples/jevdk/Sources/JevDK/AppModel.swift +++ b/examples/jevdk/Sources/JevDK/AppModel.swift @@ -124,8 +124,8 @@ final class AppModel { let verified = Set(models.map(\.repoID)) unverifiedRepos = ModelCatalog.cachedRepoIDs().filter { !verified.contains($0) } if selectedModelID == nil || selectedModel == nil { - // Default to the smallest non-MoE model of at least ~2 GB (Qwen3-4B-class, the eval's - // pick), else the smallest non-MoE one. + // Default to the smallest non-MoE model of at least ~2 GB (Qwen3-4B-class, the tested + // default), else the smallest non-MoE one. let dense = models.filter { !$0.isMoE } selectedModelID = (dense.first { ($0.sizeBytes ?? 0) >= 2_000_000_000 } ?? dense.first ?? models.first)?.id } diff --git a/examples/jevdk/Sources/JevDK/EditorView.swift b/examples/jevdk/Sources/JevDK/EditorView.swift index 4853a9c..6333d54 100644 --- a/examples/jevdk/Sources/JevDK/EditorView.swift +++ b/examples/jevdk/Sources/JevDK/EditorView.swift @@ -49,7 +49,7 @@ struct EditorView: View { .fixedSize() Text(""" - Tips from the eval: yes/no and multiple choice beat scales on small models; give \ + Tips: yes/no and multiple choice beat scales on small models; give \ options short descriptions; only ask what a model can judge from the input and \ stable general knowledge, never current facts. """) diff --git a/examples/jevdk/Sources/JevDK/Keychain.swift b/examples/jevdk/Sources/JevDK/Keychain.swift index d347e51..7997c16 100644 --- a/examples/jevdk/Sources/JevDK/Keychain.swift +++ b/examples/jevdk/Sources/JevDK/Keychain.swift @@ -4,18 +4,9 @@ import Security /// API keys for remote Jev providers, in the login Keychain. Never written to disk or logs. enum Keychain { private static let service = "JevDK" - /// The app's earlier names (AskJev, then JevLab); keys saved under them move across on first read. - private static let formerServices = ["JevLab", "AskJev"] static func get(_ account: String) -> String? { - if let value = read(account, service: service) { return value } - for former in formerServices { - guard let old = read(account, service: former) else { continue } - set(old, for: account) - delete(account, service: former) - return old - } - return nil + read(account, service: service) } static func set(_ value: String?, for account: String) { diff --git a/examples/jevdk/Sources/OpenJevKit/EditorModel.swift b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift index 4621513..37bd17d 100644 --- a/examples/jevdk/Sources/OpenJevKit/EditorModel.swift +++ b/examples/jevdk/Sources/OpenJevKit/EditorModel.swift @@ -130,7 +130,7 @@ public struct EditableQuestion: Codable, Hashable, Identifiable, Sendable { /// A named set of questions plus the local wrapper they're tuned with. What JevDK saves. public struct QuestionSet: Codable, Hashable, Sendable { - /// The SDK's `OpenJevWrapper.default` (chosen with the eval, D1b), mirrored here so this + /// The SDK's `OpenJevWrapper.default` (chosen by the SDK's evaluation), mirrored here so this /// module needs only Core. JevDK checks they match at launch. public static let defaultSystem = "You answer questions about a piece of text. Read the text, then answer the question about it. Reply with only the label of the correct answer." public static let defaultInputLabel = "Text" From 5d6a5d1690b19bcc9f67312aeb8c5d80f2294569 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 14:56:48 -0700 Subject: [PATCH 42/73] annotated-examples: jevdk excerpts and counts brought up to date Line counts now 2,627 / 2,862 (were 2,529 / 2,748). AppModel's excerpt adds importAnswerSet (DecisionAnswerSet) and open(), which now reads an exported DecisionQuestionSet back; tally re-counted. The existing excerpts were checked line by line against the current source. Co-Authored-By: Claude Opus 5.5 --- docs/annotated-examples.md | 95 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 91 insertions(+), 4 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index 152d08b..77a8282 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -41,7 +41,7 @@ non-comment lines; these examples are commented far more heavily than production | [`aiql`](#examplesaiql) | 402 | 459 | a plain-English request → one read-only SQL `SELECT` over an MCP dataset → the CSV you asked for, sandboxed SwiftUI, zero fabricated values; a pinned default model and a trust policy for the free-text picker | | [`vistanova`](#examplesvistanova) | 931 | 1,294 | a tiny local search engine: web search through a Tavily MCP server on one local model, summaries from a **pinned** MLX model on another; defends against a model that skips the tool call (7 files) | | [`mcp-chat`](#examplesmcp-chat) | 938 | 1,026 | a chat with a local model where an MCP server's own interactive view (Todoist's task list) shows up under the tool call: the conversation is `hostTranscript`, the view is `MCPAppsHost` (4 files, views excerpted) | -| [`jevdk`](#examplesjevdk) | 2,529 | 2,748 | a playground for decision-model questions: local OpenJev and hosted Jev side by side through one API, marked answers, calibration, and the tested questions exported for an app (14 files, excerpted) | +| [`jevdk`](#examplesjevdk) | 2,627 | 2,862 | a playground for decision-model questions: local OpenJev and hosted Jev side by side through one API, marked answers, calibration, and the tested questions exported for an app (14 files, excerpted) | | [`components-updates-demo`](#examplescomponents-updates-demo) | 151 | 170 | the `Components` model **onboarding**, **update** and **versions** views, driven by simulated sources so every state is reachable | | [`mlx-control-room`](#examplesmlx-control-room) | 1,434 | 1,812 | every MLX knob with a gauge, plus the supply-chain flow made visible: validate, download, **pin**, update, roll back, clean up (excerpted; UI omitted) | @@ -3783,7 +3783,7 @@ because a curated adapter lives in a different namespace. ## `examples/jevdk` -*2,529 lines of code across 14 files (2,748 with comments). Excerpted: the SDK-facing parts of +*2,627 lines of code across 14 files (2,862 with comments). Excerpted: the SDK-facing parts of `AppModel.swift` and `HostedBackends.swift` are below; the editor, batch grid, calibration sheet, model browser and results-CSV writer are plain SwiftUI and Foundation and are omitted (`// …`).* @@ -3810,7 +3810,8 @@ API keys live in the Keychain (or, for headless runs, environment variables), ne ### `Sources/JevDK/AppModel.swift` (excerpt) Runs one input on every enabled backend concurrently: hosted through `lab.decide`, local through -the OpenJev provider directly. Note there is no prompt building and no answer parsing anywhere: +the OpenJev provider directly. Then the files: the questions exported for an app, answers imported +from a CSV, and Open, which also reads an exported `.decisions.json` back. Note there is no prompt building and no answer parsing anywhere: questions go in typed, answers come back typed. ```swift @@ -3965,6 +3966,92 @@ import UniformTypeIdentifiers } } + // … + /// Load an answer set (CSV: `input`, then a column per question with the correct answer) into + /// the batch list and the marks. Inputs with line breaks are joined onto one line, since the + /// batch list is one input per line. + func importAnswerSet() { + let panel = NSOpenPanel() + panel.allowedContentTypes = [.commaSeparatedText, .plainText] + panel.message = "An answer set: a CSV with an input column and one column per question, holding the correct answer." + guard panel.runModal() == .OK, let url = panel.url else { return } + do { + let answers = try DecisionAnswerSet(csv: String(contentsOf: url, encoding: .utf8)) // ← SDK + let questions = set.questions.map(\.sdkQuestion) + let problems = answers.problems(for: questions) // ← SDK + guard problems.isEmpty else { + errorMessage = "\(url.lastPathComponent) doesn't fit these questions: " + problems.prefix(3).joined(separator: " ") + + (problems.count > 3 ? " (and \(problems.count - 3) more)" : "") + return + } + var newLabels: [String: [String: String]] = [:] + var inputs: [String] = [] + var joined = 0 + for e in answers.examples { + let one = e.input.split(whereSeparator: \.isNewline).joined(separator: " ") + if one != e.input { joined += 1 } + inputs.append(one) + var marks: [String: String] = [:] + for q in questions { + if let raw = e.answers[q.id], let key = DecisionAnswerSet.answerKey(raw, for: q) { marks[q.id] = key } // ← SDK + } + if !marks.isEmpty { newLabels[one] = marks } + } + batchText = inputs.joined(separator: "\n") + labels = newLabels + batchRows = [] + errorMessage = joined > 0 ? "Imported \(inputs.count) inputs; \(joined) had line breaks and were joined onto one line." : nil + } catch { + errorMessage = "Couldn't import: \(error.localizedDescription)" + } + } + // … + /// Open a workspace file (what Save Workspace writes), or a questions file exported for an app + /// (`.decisions.json`): its questions and system instructions, and the calibration it carries. + func open() { + let panel = NSOpenPanel() + panel.allowedContentTypes = [.json] + panel.message = "A JevDK workspace, or a questions file exported for an app (.decisions.json)." + guard panel.runModal() == .OK, let url = panel.url else { return } + do { + let data = try Data(contentsOf: url) + if let saved = try? JSONDecoder().decode(SavedSet.self, from: data) { + set = saved.set + input = saved.input ?? input + batchText = saved.batch ?? batchText + labels = saved.labels ?? [:] + calibration = saved.calibration + applyCalibration = saved.calibration != nil + } else if let exported = try? DecisionQuestionSet(json: data) { // ← SDK + let t = exported.tuning // ← SDK + set = QuestionSet(name: exported.name, system: t?.system ?? QuestionSet.defaultSystem, + inputLabel: t?.inputLabel ?? QuestionSet.defaultInputLabel, + questions: exported.questions.map(EditableQuestion.init)) + labels = [:] + if let cal = t?.calibration, let model = t?.model { + calibration = FittedCalibration( + noulTemperature: cal.noulTemperature, choiceTemperature: cal.choiceTemperature, + scoreTemperature: cal.scoreTemperature, repoID: model.rest, revision: t?.revision ?? "?", // ← SDK + system: set.system, inputLabel: set.inputLabel, fittedAt: t?.testedAt ?? Date(), samples: [:]) + } else { + calibration = nil + } + applyCalibration = calibration != nil + // An app file isn't a workspace: ⌘S asks where to save one rather than overwriting it. + results = []; batchRows = []; samples = [] + fileURL = nil + return + } else { + set = try JSONDecoder().decode(QuestionSet.self, from: data) + } + results = [] + batchRows = [] + samples = [] + fileURL = url + } catch { + errorMessage = "Couldn't open \(url.lastPathComponent): \(error.localizedDescription)" + } + } ``` ### `Sources/JevDK/HostedBackends.swift` (excerpt) @@ -4000,7 +4087,7 @@ enum APIKeys { ``` -**Tally:** 13 lines Core (`// ← SDK`), 9 Inference, 8 Remote, in the excerpts +**Tally:** 19 lines Core (`// ← SDK`), 9 Inference, 8 Remote, in the excerpts above. The decision API itself is a handful of calls (`register`/`replace(decision:)`, `route(decision:)`, `decide`, `decideWithDiagnostics`, `DecisionCalibration.fit`, `DecisionQuestionSet`, `DecisionAnswerSet`); the rest of the app is the playground around them. From 78b06c196bf14bc90672d6fff01093a593fe8d00 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 16:10:12 -0700 Subject: [PATCH 43/73] CHANGELOG (2.0): MCP resource sent verbatim; number x-mcp-header rejected From the conformance 0.2.0-alpha.12 run. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 345546e..6e47f64 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -122,7 +122,8 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) (`MCPProtocolVersion.v2026_07_28`, `.statelessPreferred`, `.minimumSupported`, `.isStateless`). - **Stateless mode** (`MCPConnectionMode.stateless(_:)`): `server/discover` instead of `initialize`, no session, the per-request `_meta` envelope, and `Mcp-Method` / `Mcp-Name` / - `Mcp-Param-*` headers. + `Mcp-Param-*` headers. A tool whose `x-mcp-header` declarations break the rules (including one + on a `number` parameter; only `string` / `integer` / `boolean` are allowed) is left out. - **`MCPVersionNegotiation`** (`.auto` / `.legacy` / `.pin(_:)`), set with `MCPSettings(versionNegotiation:)` or `MCPServerManager(versionNegotiation:)`. - **Multi-round-trip requests (MRTR).** A request that comes back `input_required` is answered @@ -141,7 +142,8 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) - Diagnostics: the `server/discover` probe and fallback, live-update status, and MRTR rounds (new category `MCP.mrtr`) are logged ([`docs/mcp-diagnostics.md`](docs/mcp-diagnostics.md)). - Authorization fixes found by the official MCP conformance suite: protected-resource metadata - at the path-suffixed then root well-known URL; `resource` must cover the server URL; metadata + at the path-suffixed then root well-known URL; `resource` must cover the server URL and is sent + exactly as the metadata publishes it (RFC 8707); metadata `issuer` and redirect `iss` are checked (RFC 8414, RFC 9207); a changed authorization server mid-connection drops the old credentials and signs in again. From a7cbae1c7a7a8e82a12c53bda5d7622dc8a97431 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 16:56:28 -0700 Subject: [PATCH 44/73] 2.0 docs: hardened, async WorkspaceAccess and the saveAs policy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CHANGELOG breaking/behavior entries; migration guide §4a (Result -> async throws, SearchResult) and §5 notes; sdk-guide §8a (how access stays inside the folder, WorkspaceLimits) and the saveAs raw/ default; annotated-examples and the aiql example/README brought to the new API. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 13 +++++++ docs/annotated-examples.md | 8 ++--- docs/migrating-to-2.0.md | 36 +++++++++++++++++++ docs/sdk-guide.md | 46 +++++++++++++++++++++--- examples/aiql/README.md | 3 +- examples/aiql/Sources/AIQL/AIQLApp.swift | 6 ++-- 6 files changed, 99 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e47f64..b681e8e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,11 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) `ToolPolicy`. `MCPServerManager.setTrustsToolAnnotations(_:server:)` / `trustsToolAnnotations(server:)` are replaced by `setTrust(_:server:)`. Mappings in the migration guide §3. +- **`WorkspaceAccess` operations are `async throws`** (`listFiles`, `readFile`, `writeFile`, + `editFile`, `deleteFile`, `search`, `tree`, `readFileRange`, `applyPatch`), not + `Result`-returning; they throw `WorkspaceError` or `CancellationError`. `search` returns + `SearchResult` (read `.matches`). The ready-made workspace tools are unchanged at the call site. + Migration guide §4a. - **New cases** on `MCPProtocolVersion` (`.v2026_07_28`) and `MCPConnectionMode` (`.stateless(_:)`); a `switch` needs `@unknown default` (Swift 6 already requires it). - **`MCPServerState`** gained fields (`trust`, `toolApproval`, `trustedToolsDigest`, @@ -60,6 +65,14 @@ RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) models; `ui://` resources are left out of model-facing resource lists. - `languageModelSession` is documented as the escape hatch: run turns with `respond` / `streamResponse`. +- **Workspace file access is hardened.** Files are reached by descriptor from the granted folder + and no symlink is followed out of it — including mid-operation (1.x `search` followed a symlink + inside the folder to a file outside it). Reads, searches and edits are bounded by + `WorkspaceLimits` and stream instead of loading whole files; `search` reports files it skipped. + `deleteFile` removes single files only (1.x deleted folders recursively) and deletes a symlink + itself, not its target. `editFile` keeps file permissions. Writes are temp-then-rename. +- **`FileBackedTool` `saveAs` writes under `raw/` by default**, data-file extensions only, no + hidden paths (`SaveAsPolicy`; `.unrestricted` restores 1.x). The receipt gives the real path. - **Local (MLX) models report real usage**: input and output token counts are the model's own (input was always 0). `languageModelSession.usage` sums every model call in the session; one turn's figures are on its response's `usage`, with `mlx.*` metadata (`MLXUsageMetadataKey`). diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index 77a8282..281a75b 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -2433,7 +2433,7 @@ there's only ever one query call. `WHERE` / `BETWEEN` / `ORDER BY … LIMIT` / ` SQLite. In 2.0 the data tools come from **`lab.mcp`**: the app adds the server there, turns on its four most data-like tools, and sets `lab.mcp.setFileBackedOutput(MCPFileBackedOutput(root:…), server:)` for the run's folder, so `makeSession(includeMCPTools: true)` offers each with a `saveAs` argument -that writes its result to a file. Links `LocalLMLabSDKInference` for the MLX model. Because the model +that writes its result to a file — under `raw/`, data-file types only (the default `SaveAsPolicy`). Links `LocalLMLabSDKInference` for the MLX model. Because the model field is free text, the app can't pin everything ahead of time, so it layers three supply-chain controls: the default model is **pinned** to a reviewed commit (`pinnedRevisions:`), any other model is pinned on first download (`MLXFilePinStore`), and a **trust policy** (`MlxCommunityOnly`) limits @@ -2698,15 +2698,15 @@ final class AIQLModel: ObservableObject { // The result CSV. Instructions ask for "out.csv"; if a weak model wrote it somewhere // else, fall back to the most recently written .csv. var name = "out.csv" - if case .failure = WorkspaceAccess.readFile(in: root, path: "out.csv"), // ← SDK - case .success(let entries) = WorkspaceAccess.listFiles(in: root, subpath: nil) { // ← SDK + if (try? await WorkspaceAccess.readFile(in: root, path: "out.csv")) == nil, // ← SDK (async throws in 2.0) + let entries = try? await WorkspaceAccess.listFiles(in: root, subpath: nil) { // ← SDK if let newest = entries .filter({ !$0.isDirectory && $0.name.hasSuffix(".csv") }) .max(by: { ($0.modifiedDate ?? .distantPast) < ($1.modifiedDate ?? .distantPast) }) { name = newest.name } } - guard case .success(let csv) = WorkspaceAccess.readFile(in: root, path: name) else { // ← SDK + guard let csv = try? await WorkspaceAccess.readFile(in: root, path: name) else { // ← SDK return .failed("The model finished but didn't write a spreadsheet. Try rephrasing the request, or a larger model.") } step("Saved \(name).") diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md index ab9bf4a..553cb0a 100644 --- a/docs/migrating-to-2.0.md +++ b/docs/migrating-to-2.0.md @@ -55,6 +55,32 @@ restore servers through `restore(from:)` instead. Full walkthrough: are non-frozen; Swift 6 already requires `@unknown default` in a `switch` over them, so this only affects code that silenced that error another way. +## 4a. Workspace file access is `async throws` + +Every `WorkspaceAccess` operation now `throws` instead of returning a `Result`, and is `async` +(it runs off your actor and can be cancelled): + +```swift +// 1.x +switch WorkspaceAccess.readFile(in: root, path: "notes.md") { +case .success(let text): show(text) +case .failure(let error): show(error.message) +} + +// 2.0 +do { + show(try await WorkspaceAccess.readFile(in: root, path: "notes.md")) +} catch let error as WorkspaceAccess.WorkspaceError { + show(error.message) +} +``` + +`try WorkspaceAccess.x(…).get()` becomes `try await WorkspaceAccess.x(…)`. `search` returns a +`SearchResult`: the hits are `.matches`, and `.skippedTooLarge` / `.scanBudgetExhausted` say what +wasn't searched. If you only use the ready-made tools (`ReadWorkspaceFileTool`, +`SearchWorkspaceTool`, …), nothing changes. Every operation and tool also takes an optional +`limits: WorkspaceLimits` ([`sdk-guide.md` §8a](sdk-guide.md)). + ## 5. What behaves differently - **A session's MCP tools follow `lab.mcp` between turns.** Turn a tool on, add a server, or @@ -78,6 +104,16 @@ affects code that silenced that error another way. - **A local-model prompt that can't fit** (over the context window, or the GPU memory left) is refused before it runs, as `LocalLMLabError.context`. With `retryOnContextOverflow` set, it's compacted and retried like any overflow. +- **Workspace access never follows a symlink out of the granted folder**, even one swapped in + mid-operation, and `search`, `tree` and `listFiles` don't follow symlinks at all. Big files are + read in bounded pieces: `readFile` returns the head of a file over 8 MiB with a pointer to + `readFileRange`; `search` skips files over 64 MiB and lists them. `deleteFile` refuses folders + (1.x deleted them recursively) and removes a symlink rather than its target. `editFile` matches + byte-for-byte and keeps the file's permissions. +- **`saveAs` lands under `raw/`.** A model asking for `data.json` gets `raw/data.json` (the receipt + says so); paths outside `raw/`, non-data extensions and hidden paths are refused. For another + folder: `MCPFileBackedOutput(root:…, saveAsPolicy: SaveAsPolicy(directory: "exports"))`; for the + 1.x behaviour, `.unrestricted`. - **`contextBudget` measures the latest turn**, not the whole session (1.0's `fractionUsed` grew every turn). It's recorded by `respond` / `streamResponse`; turns run on `languageModelSession` directly aren't counted. diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index d22bdd0..b73a89f 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -2470,10 +2470,46 @@ Required entitlements for this to work under App Sandbox: Once you have a resolved, access-bracketed root `URL` from the pattern above, `WorkspaceAccess` — an ordinary Core type, not a permission-gated connector — is what actually -reads and writes inside it: `listFiles`/`readFile`/`writeFile`/`editFile`/`deleteFile`, each -scoped to the root with the same symlink-escape check the picker pattern itself doesn't need to -worry about. No `requestAccess()` here — there's no OS dialog for this, the picker *is* the -consent, entirely on your side. +reads and writes inside it: `listFiles`/`readFile`/`writeFile`/`editFile`/`deleteFile` (plus +`search`, `tree`, `readFileRange`, `applyPatch`). No `requestAccess()` here — there's no OS dialog +for this, the picker *is* the consent, entirely on your side. + +Every operation is `async throws` (2.0): it throws `WorkspaceAccess.WorkspaceError` (its `message` +is written for the model) or `CancellationError`, and runs off your actor, so calling it from the +main actor doesn't block the UI: + +```swift +do { + let text = try await WorkspaceAccess.readFile(in: root, path: "README.md") + try await WorkspaceAccess.editFile(in: root, path: "README.md", + oldString: "v1.0", newString: "v2.0", replaceAll: false) +} catch let error as WorkspaceAccess.WorkspaceError { + print(error.message) +} +``` + +**How it stays inside the folder.** A path is resolved once, to real (symlink-free) components +under the root. From there every file is reached by descriptor: the root is opened, each folder +below it is opened with `openat(… O_NOFOLLOW)`, and the file is checked through its own descriptor. +A symlink that points outside the folder is refused, and so is one swapped in mid-operation — +the race the 1.x path-based checks left open. A symlink whose target stays inside the folder still +works for `readFile`/`writeFile`/`editFile`; `search`, `tree` and `listFiles` never follow symlinks. +Writes go to a temp file that is renamed into place (a create never overwrites a file that +appeared in the meantime), `editFile` keeps the file's permissions and refuses to commit if the +file changed while it was being edited, and `deleteFile` removes single files only — never a +folder, and for a symlink, the link rather than its target. What this can't stop: another local +process that can write to the folder changing a file's *contents* between two of your calls. Treat +the folder as a trust boundary shared with everything that can write to it. + +**How much work one call can do.** `WorkspaceLimits` bounds it; every operation and every +workspace tool takes `limits:`. Defaults: `readFile` reads at most 8 MiB off disk (`maxReadBytes`) +and returns at most 2,000,000 characters (`maxReturnedCharacters`), cutting at a character +boundary and pointing the model at `readFileRange`; `editFile`, `applyPatch` and `search` won't +process a file over 64 MiB (`maxFileBytes`) — search skips it and says so in +`SearchResult.skippedTooLarge`; one `search` or `readFileRange` reads at most 256 MiB in total +(`maxScanBytes`, `SearchResult.scanBudgetExhausted`); each line given to a regular expression is +capped at 8 KiB (`maxRegexInputBytes` — `NSRegularExpression` can't be cancelled, so bounding its +input is the only thing that bounds its work); `writeFile` accepts at most 64 MiB (`maxWriteBytes`). `editFile` — the one write operation actually meant for AI-assisted modification of an existing file — is search-and-replace, not a unified-diff/patch format: `oldString`/`newString`, and it @@ -2525,7 +2561,7 @@ invents the ones it didn't see. **`FileBackedTool`** wraps a dynamic-schema tool (an MCP tool adapter is the motivating case) and adds one root-level argument, `saveAs`. When the model supplies a path, the wrapped tool's -raw result is written to `/` and only a short receipt — byte/line count and a +raw result is written to `/raw/` (the default `SaveAsPolicy`) and only a short receipt — byte/line count and a bounded head preview — returns. The model then works from the file (`readFileRange` a window → the verbs below), and the payload never enters its context. It's not a `Tool` that calls another `Tool` (the model can't invoke that) — it's a **decorator the host applies** when diff --git a/examples/aiql/README.md b/examples/aiql/README.md index dc1ce4b..0b4237a 100644 --- a/examples/aiql/README.md +++ b/examples/aiql/README.md @@ -49,7 +49,8 @@ evaluates a condition, or handles a value. - which tools the model even sees: the server's tools are ranked by how "dataset-like" the name looks and only the top few are enabled; the raw-file reader is withheld - the raw payload's path — `FileBackedTool`'s `saveAs` parks it in `raw/data.json`; it never - enters the model's context + enters the model's context. Since SDK 2.0 that's enforced, not just asked for: the default + `SaveAsPolicy` keeps `saveAs` under `raw/`, data-file types only, no hidden paths **The model decides** — a data-tool pick, a table name, then one `SELECT`: diff --git a/examples/aiql/Sources/AIQL/AIQLApp.swift b/examples/aiql/Sources/AIQL/AIQLApp.swift index e02291a..3ecf5ec 100644 --- a/examples/aiql/Sources/AIQL/AIQLApp.swift +++ b/examples/aiql/Sources/AIQL/AIQLApp.swift @@ -289,15 +289,15 @@ final class AIQLModel: ObservableObject { // The result CSV. Instructions ask for "out.csv"; if a weak model wrote it somewhere // else, fall back to the most recently written .csv. var name = "out.csv" - if case .failure = WorkspaceAccess.readFile(in: root, path: "out.csv"), - case .success(let entries) = WorkspaceAccess.listFiles(in: root, subpath: nil) { + if (try? await WorkspaceAccess.readFile(in: root, path: "out.csv")) == nil, + let entries = try? await WorkspaceAccess.listFiles(in: root, subpath: nil) { if let newest = entries .filter({ !$0.isDirectory && $0.name.hasSuffix(".csv") }) .max(by: { ($0.modifiedDate ?? .distantPast) < ($1.modifiedDate ?? .distantPast) }) { name = newest.name } } - guard case .success(let csv) = WorkspaceAccess.readFile(in: root, path: name) else { + guard let csv = try? await WorkspaceAccess.readFile(in: root, path: name) else { return .failed("The model finished but didn't write a spreadsheet. Try rephrasing the request, or a larger model.") } step("Saved \(name).") From a7600c95747ffb1f47186fd33259aa7f6eaeb502 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 17:01:35 -0700 Subject: [PATCH 45/73] Docs: calibrate in JevDK, use the calibration in your app The docs said how to fit a calibration but not clearly how it reaches an app. Now each says: fit it in JevDK, then either Export Questions (the .decisions.json carries it, and OpenJevDecisionProvider(mlx:tunedWith:) applies it) or Copy code (the three values plus the system instructions they were fitted with, for init(mlx:wrapper:calibration:)). No fitting code in the app; refit and re-export when the model or instructions change. Co-Authored-By: Claude Opus 5.5 --- docs/sdk-guide.md | 23 +++++++++++++++++++---- examples/jevdk/GUIDE.md | 32 ++++++++++++++++++++++++++++++-- examples/jevdk/README.md | 5 +++++ 3 files changed, 54 insertions(+), 6 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index b73a89f..88187f8 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1924,14 +1924,29 @@ ones are read off an ordinary language model — how likely it was to say each o and on a small model they're often 100% even when it's wrong. Don't put tight thresholds on uncalibrated `.tokenScored` answers. -For a local decider you fit a **calibration** on your own marked answers (JevDK's **Calibrate…** -does it) and pass it in; `fidelity` then says `.tokenScored(calibrated: true)`. It holds for one -model, revision, wrapper and kind of question, so the SDK ships no fitted values. +For a local decider you fit a **calibration** on your own marked answers and pass it in; +`fidelity` then says `.tokenScored(calibrated: true)`. It holds for one model, revision, wrapper +and kind of question, so the SDK ships no fitted values. **The usual way is to fit it in JevDK +and use the result in your app**, with no fitting code of your own: + +```swift +// 1. In JevDK: mark answers, Calibrate… → Fit calibration, then File → Export Questions…. +// The .decisions.json carries the calibration with the questions it was fitted for. +let set = try DecisionQuestionSet(contentsOf: url) +let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // applies set.tuning?.calibration + +// 2. Or copy the snippet (Calibrate sheet → Copy code): the values plus the system instructions +// they were fitted with — the calibration only holds with the same wrapper. +let openjev2 = OpenJevDecisionProvider(mlx: mlx, + wrapper: OpenJevWrapper(system: "…", inputLabel: "…"), + calibration: DecisionCalibration(noulTemperature: 9.11, choiceTemperature: 0.25, scoreTemperature: 1.0)) +``` + +To fit in code instead (for example from `lab.evaluate`'s `samples`): ```swift let cal = DecisionCalibration.fit(samples) // [DecisionCalibration.Sample], from marked answers cal.measure(samples) // accuracy, expectedCalibrationError, logLoss -let openjev = OpenJevDecisionProvider(mlx: mlx, calibration: cal) ``` Calibration is temperature scaling per question kind: it never changes which answer wins, only diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index d781227..b5ebd12 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -229,8 +229,36 @@ only use the top answer. and gets sharpened. 3. Read the before → after table. Lower is better on both measures (see section 6). 4. Turn on **Show local results calibrated** to see what your app will see. -5. **Export Questions…** (step 8) carries it into your app with the questions. If you change the - model or the system instructions later, JevDK flags the calibration until you redo it. +5. If you change the model or the system instructions later, JevDK flags the calibration until + you redo it. + +**Use the calibration in your app.** You fit it in JevDK; your app only applies it. Two ways: + +- **With the questions (recommended):** **File → Export Questions…** writes the calibration into + the `.decisions.json` file, next to the questions, model and system instructions it was fitted + for. Load that file in your app and build the decider from it; the calibration is applied to + every answer (step 8): + + ```swift + let set = try DecisionQuestionSet(contentsOf: url) + let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // calibration included + ``` + +- **Without the file:** in the Calibrate sheet, **Copy code** copies a snippet with the three + fitted values and the system instructions they were fitted with. Paste it into your app; keep + the system instructions too, since the calibration only holds with them: + + ```swift + let openjev = OpenJevDecisionProvider(mlx: mlx, + wrapper: OpenJevWrapper(system: "…", inputLabel: "…"), // as tested in JevDK + calibration: DecisionCalibration(noulTemperature: 9.11, choiceTemperature: 0.25, scoreTemperature: 1.0)) + ``` + +Either way, your app's answers then come back with `fidelity == .tokenScored(calibrated: true)`. +A calibration holds only for the model version, system instructions and kinds of question it was +fitted on: refit in JevDK and re-export when any of them changes. `openjev.tuningMismatch(for:)` +tells your app when the model on the device isn't the one you calibrated. Hosted deciders don't +use it: TypeSafe's Jev is calibrated already. What calibration can't do: fix wrong answers. In our test, a confidently wrong answer stayed wrong; it just became less confidently wrong. Get the questions right first (steps 4 and 5). diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index c1e1cc0..a57e3f4 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -127,6 +127,11 @@ for your app: 4. Turn on **Show local results calibrated** to see what your app will see. It's saved with the workspace (⌘S). If you change the model or the system instructions, JevDK flags the calibration until you refit. +5. **Use it in your app.** **File → Export Questions…** puts the calibration in the + `.decisions.json` file with the questions; `OpenJevDecisionProvider(mlx:tunedWith:)` applies it. + Or **Copy code** in the Calibrate sheet: a snippet with the three values and the system + instructions they were fitted with, for `OpenJevDecisionProvider(mlx:wrapper:calibration:)`. See + [GUIDE.md](GUIDE.md) step 7. ## Taking it to your app From 05968a964fe1d2eaf01f58fe6a880eb1baa3bd6e Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 17:05:13 -0700 Subject: [PATCH 46/73] Docs: where the three calibration numbers come from (X, Y, Z) A marked screenshot of JevDK's Calibrate sheet: the Temperature column of the Fitted table (X yes/no, Y choice, Z score; a kind with no questions stays 1.0) and the same values in the Copy code snippet. Code examples use X, Y, Z placeholders to match. Co-Authored-By: Claude Opus 5.5 --- docs/sdk-guide.md | 3 ++- examples/jevdk/GUIDE.md | 7 ++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 88187f8..4665d7c 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1939,7 +1939,8 @@ let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // applies // they were fitted with — the calibration only holds with the same wrapper. let openjev2 = OpenJevDecisionProvider(mlx: mlx, wrapper: OpenJevWrapper(system: "…", inputLabel: "…"), - calibration: DecisionCalibration(noulTemperature: 9.11, choiceTemperature: 0.25, scoreTemperature: 1.0)) + calibration: DecisionCalibration(noulTemperature: X, choiceTemperature: Y, scoreTemperature: Z)) +// X, Y, Z: the Temperature column of JevDK's Calibrate sheet (yes/no, choice, score; 1.0 = no change). ``` To fit in code instead (for example from `lab.evaluate`'s `samples`): diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index b5ebd12..aa309eb 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -251,9 +251,14 @@ only use the top answer. ```swift let openjev = OpenJevDecisionProvider(mlx: mlx, wrapper: OpenJevWrapper(system: "…", inputLabel: "…"), // as tested in JevDK - calibration: DecisionCalibration(noulTemperature: 9.11, choiceTemperature: 0.25, scoreTemperature: 1.0)) + calibration: DecisionCalibration(noulTemperature: X, choiceTemperature: Y, scoreTemperature: Z)) ``` + The three numbers are the **Temperature** column of the Calibrate sheet's **Fitted** table: **X** + for yes/no (Noul), **Y** for Choice and **Z** for Score. A kind you have no questions of shows no + row and stays at 1.0, which means "no change". + + Either way, your app's answers then come back with `fidelity == .tokenScored(calibrated: true)`. A calibration holds only for the model version, system instructions and kinds of question it was fitted on: refit in JevDK and re-export when any of them changes. `openjev.tuningMismatch(for:)` From e35e4b5db3507da91aa2eb27782bdeb4ceac82bc Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 17:08:44 -0700 Subject: [PATCH 47/73] Docs: say where the calibration screenshot comes from MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The X/Y/Z screenshot now has a "where to find them" line (in JevDK: Batch, mark answers, Calibrate…, Fit calibration) and a caption that names the window and its two parts, for readers who haven't used JevDK. "Calibrate sheet" is now "Calibrate window" throughout. Co-Authored-By: Claude Opus 5.5 --- docs/sdk-guide.md | 4 ++-- examples/jevdk/GUIDE.md | 4 ++-- examples/jevdk/README.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 4665d7c..7465ca6 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1935,12 +1935,12 @@ and use the result in your app**, with no fitting code of your own: let set = try DecisionQuestionSet(contentsOf: url) let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // applies set.tuning?.calibration -// 2. Or copy the snippet (Calibrate sheet → Copy code): the values plus the system instructions +// 2. Or copy the snippet (Calibrate window → Copy code): the values plus the system instructions // they were fitted with — the calibration only holds with the same wrapper. let openjev2 = OpenJevDecisionProvider(mlx: mlx, wrapper: OpenJevWrapper(system: "…", inputLabel: "…"), calibration: DecisionCalibration(noulTemperature: X, choiceTemperature: Y, scoreTemperature: Z)) -// X, Y, Z: the Temperature column of JevDK's Calibrate sheet (yes/no, choice, score; 1.0 = no change). +// X, Y, Z: the Temperature column of JevDK's Calibrate window (yes/no, choice, score; 1.0 = no change). ``` To fit in code instead (for example from `lab.evaluate`'s `samples`): diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index aa309eb..21355f2 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -244,7 +244,7 @@ only use the top answer. let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // calibration included ``` -- **Without the file:** in the Calibrate sheet, **Copy code** copies a snippet with the three +- **Without the file:** in the Calibrate window, **Copy code** copies a snippet with the three fitted values and the system instructions they were fitted with. Paste it into your app; keep the system instructions too, since the calibration only holds with them: @@ -254,7 +254,7 @@ only use the top answer. calibration: DecisionCalibration(noulTemperature: X, choiceTemperature: Y, scoreTemperature: Z)) ``` - The three numbers are the **Temperature** column of the Calibrate sheet's **Fitted** table: **X** + The three numbers are the **Temperature** column of the Calibrate window's **Fitted** table: **X** for yes/no (Noul), **Y** for Choice and **Z** for Score. A kind you have no questions of shows no row and stays at 1.0, which means "no change". diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index a57e3f4..7432233 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -129,7 +129,7 @@ for your app: calibration until you refit. 5. **Use it in your app.** **File → Export Questions…** puts the calibration in the `.decisions.json` file with the questions; `OpenJevDecisionProvider(mlx:tunedWith:)` applies it. - Or **Copy code** in the Calibrate sheet: a snippet with the three values and the system + Or **Copy code** in the Calibrate window: a snippet with the three values and the system instructions they were fitted with, for `OpenJevDecisionProvider(mlx:wrapper:calibration:)`. See [GUIDE.md](GUIDE.md) step 7. From 64772ffa38d1231fee4f47465c97c8ea134ccc4c Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 17:14:27 -0700 Subject: [PATCH 48/73] Docs: calibration in your app, explained in plain English first The "calibrate in JevDK, use it in your app" text assumed readers knew JevDK and the SDK. It now leads with what a calibration is (three numbers that tone a local decider's confidence down or up, one per kind of question, worked out once in JevDK), then how to get them, then two plain options: let them travel with the exported questions file, or paste Copy code's code. API names follow as a footnote, not as the explanation. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/GUIDE.md | 35 ++++++++++++++++------------------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index 21355f2..dd7bc7d 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -232,21 +232,23 @@ only use the top answer. 5. If you change the model or the system instructions later, JevDK flags the calibration until you redo it. -**Use the calibration in your app.** You fit it in JevDK; your app only applies it. Two ways: +**Use the calibration in your app.** A calibration is just three numbers: the **Temperature** +column of the Calibrate window's **Fitted** table, **X** for yes/no questions, **Y** for choice and +**Z** for score (a kind you have no questions of stays at 1.0, "no change"). Your app doesn't +work them out again; it only needs to be given them. Two ways: -- **With the questions (recommended):** **File → Export Questions…** writes the calibration into - the `.decisions.json` file, next to the questions, model and system instructions it was fitted - for. Load that file in your app and build the decider from it; the calibration is applied to - every answer (step 8): +- **Let them travel with your questions (easiest).** **File → Export Questions…** saves your + questions as a file for your app, and that file includes the three numbers. Add the file to + your app and set up the decider from it; the numbers are used automatically (step 8): ```swift let set = try DecisionQuestionSet(contentsOf: url) - let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // calibration included + let openjev = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // the numbers come with the file ``` -- **Without the file:** in the Calibrate window, **Copy code** copies a snippet with the three - fitted values and the system instructions they were fitted with. Paste it into your app; keep - the system instructions too, since the calibration only holds with them: +- **Put them in your code yourself.** **Copy code** in the Calibrate window copies ready-to-paste + code with X, Y and Z filled in, plus the system instructions you tested with. Paste it where + your app sets up its decider, and keep the instructions: the numbers only fit them. ```swift let openjev = OpenJevDecisionProvider(mlx: mlx, @@ -254,16 +256,11 @@ only use the top answer. calibration: DecisionCalibration(noulTemperature: X, choiceTemperature: Y, scoreTemperature: Z)) ``` - The three numbers are the **Temperature** column of the Calibrate window's **Fitted** table: **X** - for yes/no (Noul), **Y** for Choice and **Z** for Score. A kind you have no questions of shows no - row and stays at 1.0, which means "no change". - - -Either way, your app's answers then come back with `fidelity == .tokenScored(calibrated: true)`. -A calibration holds only for the model version, system instructions and kinds of question it was -fitted on: refit in JevDK and re-export when any of them changes. `openjev.tuningMismatch(for:)` -tells your app when the model on the device isn't the one you calibrated. Hosted deciders don't -use it: TypeSafe's Jev is calibrated already. +Afterwards, your app's answers say they're calibrated (`fidelity == .tokenScored(calibrated: true)`). +The numbers fit the model version, system instructions and kinds of question you tested, so fit +them again and re-export when any of those changes; `openjev.tuningMismatch(for:)` tells your app +when the model on the device isn't the one you calibrated. Hosted deciders don't need this: +TypeSafe's Jev comes calibrated. What calibration can't do: fix wrong answers. In our test, a confidently wrong answer stayed wrong; it just became less confidently wrong. Get the questions right first (steps 4 and 5). From ab21a79f8a736202796e320f23d7a426093939b6 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 19:42:34 -0700 Subject: [PATCH 49/73] Docs: decisions that need your product's knowledge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A decider sees only the input and the questions, so routing that depends on knowledge about the app (which component a bug is in, who owns it) needs a pattern. Best first: classify the component and map component -> team in code; put the knowledge in the option descriptions; send the app's own context with the input as JSON; retrieve the relevant part first for large knowledge. Not in the local-only system instructions. In the JevDK guide (with code), jev.html (plain English) and SDK guide §6c (short). Co-Authored-By: Claude Opus 5.5 --- docs/sdk-guide.md | 21 ++++++++++++++++ examples/jevdk/GUIDE.md | 56 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 7465ca6..9a337d6 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -2042,6 +2042,27 @@ let result = try await lab.evaluate(route: "support", questions: set.questions, when wrong (a big gap means a threshold can catch the misses), calibration error, and `samples` for `DecisionCalibration.fit`. +### Decisions that need your product's knowledge + +A decider sees only `state` and the questions (there's no system prompt), so knowledge it needs, +such as which part of your software a bug report concerns and who owns it, has to come through +those, or stay in your code: + +- **Classify, then map in code** (usually best): ask which *component* the input is about, with the + product knowledge in each option's description, and map component → team with an ordinary table. + Ownership changes then touch the table, not the questions or the calibration. +- **Knowledge in the options:** describe each team by what it owns (`"payments": "billing, + invoices, the Checkout module"`). Up to 26 options locally, 255 hosted. +- **Knowledge in the input:** send the context your app already has as JSON alongside the input, + `try .encoding(Ticket(report:screen:stackTraceModules:))`. Keep it short (`maxStateCharacters` + on hosted backends; longer inputs are slower locally). +- **Retrieve first:** for a large knowledge base, select the relevant passages in your app (or with + your chat model) and send only those. + +Don't put the knowledge in an `OpenJevWrapper`'s system text: hosted backends never see it, and a +calibration only holds for the wrapper it was fitted with. Full walkthrough with code: +[JevDK's guide](../examples/jevdk/GUIDE.md#when-the-decision-needs-your-products-knowledge). + ### Writing questions that work - Only ask what a model can judge from the input and general knowledge — **never current diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index dd7bc7d..1109519 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -71,6 +71,62 @@ Poor fits: - **Judgements even people would disagree on.** If two colleagues wouldn't give the same answer, the model won't be consistent either. +### When the decision needs your product's knowledge + +Say the decider routes bug reports to teams. To do that it has to know something about your +software: which parts exist, and who owns what. It only knows what's in the **input** and the +**questions**. There's no system prompt to put background in: hosted deciders don't have one, +and the SDK leaves it out so local and hosted behave the same. So the knowledge has to reach it +one of these ways, best first: + +1. **Ask what the report is about, and keep "who owns what" in your code.** The decider is good at + recognising which part of the product a report describes; it shouldn't need to know your org + chart. Ask about components, then look the team up in an ordinary table: + + ```swift + let d = try await lab.decide(route: "triage", state: .text(report), questions: [ + .choice("component", "Which part of the product does this bug report describe?", criteria: [ + "checkout": "cart, payment form, Stripe errors, order confirmation", + "sync": "files not syncing, conflicts, the desktop sync client", + "auth": "sign-in, password reset, SSO, two-factor codes", + "editor": "the document editor, formatting, comments, undo", + ]), + ]) + let team = owners[d.choice("component")!.choice] // your table: component → team + ``` + + When teams reorganise, you change the table, not the questions, so there's nothing to re-test + or recalibrate. The product knowledge lives in the **option descriptions**: write them in the + words users actually use ("Stripe errors", "files not syncing"). +2. **Describe each team by what it owns.** If teams map neatly onto areas, make the teams the + options: `"payments": "billing, invoices, the Checkout module, Stripe integration"`. Fine while + each description stays short. A local decider takes up to 26 options; hosted Jev up to 255. +3. **Send the knowledge with the input.** When a line per option isn't enough, send structured + context alongside the report, as JSON: + + ```swift + struct Ticket: Encodable { let report: String; let screen: String; let stackTraceModules: [String] } + let d = try await lab.decide(route: "triage", state: try .encoding(ticket), questions: [...]) + ``` + + The best context is what your app already knows as data: the screen the user was on, the app + version, the modules in a stack trace. It's precise and free. Keep it short: a local decider + reads the input once for all the questions in a call, but a long input is still slower, and + hosted services cap the size (6,000 characters on the Featherless demo). +4. **Look up the relevant part first.** For a big knowledge base (an architecture doc, a wiki), + don't send all of it. Have your app find the few relevant pieces (match file paths or error + codes, or search your docs) and send only those. If finding them is itself a judgement, let + your chat model do that step and give the decider its short summary. + +Don't put product knowledge in JevDK's **System instructions**: only the local decider sees them, +and calibration is tied to their exact words. And don't ask questions whose answer isn't in the +input or the options ("Which team owns the sync engine?"); like current facts, the decider can +only judge what it's given. + +**Test it in JevDK** as usual: real reports in **Batch**, the knowledge in the option descriptions +or pasted into each input the way your app will send it, the right component or team marked. +Compare the approaches by their ✓s on your own reports. + --- ## 3. The five things that decide your results From 51680578a811719324182aa43daf0a9fb5bdf0e5 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 20:20:54 -0700 Subject: [PATCH 50/73] docs: JevWire in the CHANGELOG and the SDK guide's reference Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 2 ++ docs/sdk-guide.md | 13 +++++++++++++ 2 files changed, 15 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b681e8e..11e16e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -194,6 +194,8 @@ API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdec → `DecisionEvaluation` (accuracy per question, misses, confidence when right and wrong) for regression tests in CI; `DecisionCalibration` (temperature scaling per question kind: `fit`, `measure`, `apply`). +- **`JevWire`** (Core): hosted Jev's JSON both ways (requests and responses, order kept), for + serving decisions over HTTP as well as calling hosted Jev; `JevDecisionProvider` is built on it. - **`LocalLMLabSDKInference` (macOS 27):** `OpenJevDecisionProvider` — answers each question in one forward pass over the allowed labels' probabilities, no generation, the input read once for all questions (about 150 ms for four questions on Qwen3 4B). `OpenJevWrapper` (+ `.default`), diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 9a337d6..73a9a2b 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -3549,6 +3549,19 @@ struct JevProviderConfig { // scheme, displayName, endpoint, auth, models, all func validated() throws -> JevProviderConfig } +// --- hosted Jev's wire format (both directions; a server can answer the same API) --- +enum JevWire { + struct Request { var model: String?; var request: DecisionRequest } + struct Response { var model: String?; var answers: [String: DecisionAnswer]; var usage: DecisionUsage? + var fidelity: DecisionFidelity?; var errorMessage: String? } + static func encodeRequest(_ request: DecisionRequest, model: String) throws -> Data + static func decodeRequest(_ data: Data) throws -> Request // order-preserving + static func encodeResponse(_ decision: Decision, model: String, request: DecisionRequest) throws -> Data + static func decodeResponse(_ data: Data, request: DecisionRequest) throws -> Response + static func encodeError(message: String, type: String, code: Int) -> Data + static func errorMessage(in data: Data) -> String? +} + // --- calibration, question sets, answer sets, evaluation --------------------------- struct DecisionCalibration: Codable { var noulTemperature, choiceTemperature, scoreTemperature: Double; static let none From 4a4c9bc79961fab6478a3bc13e0244c8b4227e7e Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 4 Oct 2026 20:24:53 -0700 Subject: [PATCH 51/73] JevDK: Export Server Config, and model versions shown (jev-serve S0b) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - File → Export Server Config… (For your app) writes or updates a jev-serve config (JevServeConfig in OpenJevKit): listen address, optional token (offered when the file is new), and this question set's model, version, system instructions and calibration as one entry; other entries are kept. Written readable only by the user. - A green confirmation after Export Questions / Export Server Config names the model, the version it's pinned to, and whether it's calibrated. - The model's version (Hugging Face commit, short) shows in Models, the model tooltip, the status bar and the Calibrate window; an out-of-date calibration names both versions. - README Files section: five kinds of file; a note on model versions. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/GUIDE.md | 4 +- examples/jevdk/README.md | 8 +- examples/jevdk/Sources/JevDK/AppModel.swift | 72 ++++++++++++++- .../jevdk/Sources/JevDK/Calibration.swift | 4 +- .../Sources/JevDK/CalibrationSheet.swift | 2 +- examples/jevdk/Sources/JevDK/JevDKApp.swift | 8 +- .../jevdk/Sources/JevDK/ModelsSheet.swift | 2 +- .../jevdk/Sources/JevDK/ResultsView.swift | 13 +++ .../Sources/OpenJevKit/JevServeConfig.swift | 87 +++++++++++++++++++ 9 files changed, 191 insertions(+), 9 deletions(-) create mode 100644 examples/jevdk/Sources/OpenJevKit/JevServeConfig.swift diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index 1109519..2b109d5 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -434,7 +434,9 @@ Sixteen messages are enough to show the method, not to choose a model. Use 50 or imports and exports it as CSV (**File → Import / Export Answers**). - **Workspace:** JevDK's own file (⌘S / ⌘O) with everything you're working on. Not for your app; for that, **Export Questions**. The README's [Files](README.md#files) section compares - the four kinds of file. + the five kinds of file. +- **Model version:** the exact Hugging Face commit of a model (shown as e.g. `4dcb3d1`). Exports + record it, so what ships is what you tested. - **Backend:** where the decider runs: locally (OpenJev on this Mac) or hosted (Featherless, TypeSafe). - **System instructions / wrapper:** the text that frames every question for a local model. diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index 7432233..86956a3 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -75,12 +75,13 @@ to try a second wording side by side. **⌘S** saves the whole workspace to a fi ## Files -JevDK works with four kinds of file. The File menu groups them: +JevDK works with five kinds of file. The File menu groups them: | File menu | The file | What's in it | Use it to | |---|---|---|---| | **Workspace:** Open…, Save Workspace (⌘S), Save Workspace As… | `Customer support.jevdk.json`, JevDK's own format | Everything: questions, system instructions, the input and batch list, your marked answers, the calibration | Pick your work back up, or hand it to a colleague. JevDK also keeps the last workspace between launches on its own. | | **For your app:** Export Questions… (⌘E) | `customer-support.decisions.json`, the SDK's `DecisionQuestionSet` | The questions exactly as tested, plus the model, version, system instructions and calibration they were tested with. No inputs or marks. | Bundle it in your app; `OpenJevDecisionProvider(mlx:tunedWith:)` loads it. | +| **For your app:** Export Server Config… | `jev-serve.json` | Where jev-serve listens, an optional token, and the models it serves, each with its version, system instructions and calibration. Exporting again adds or replaces this question set's entry. Readable only by you. | Serve the tested setup over hosted Jev's HTTP API on this Mac, with the jev-serve example. | | **Test data:** Import Answers (CSV)…, Export Answers (CSV)… | `customer-support.answers.csv` | Your batch inputs and the correct answers you marked: an `input` column, then one column per question | Edit test cases in a spreadsheet, share them, and run them in your app's tests with `lab.evaluate`. | | **Test data:** Append Run to Results CSV… | `jevdk-results.csv`, grows with every run | Every answer from every backend in a run, with the model, timing and whether it was right | Compare models and wordings over time (below). | @@ -91,6 +92,11 @@ on (its questions, system instructions and calibration; it has no inputs or mark The Batch toolbar has the same answer and results actions (**Answers**, **Results CSV**). +**Model versions.** A model on Hugging Face can change. JevDK shows the exact version you're +testing (a short commit id, e.g. `4dcb3d1`) in **Models**, the status bar and the Calibrate +window, and every export records it, so your app or jev-serve uses (and downloads) exactly that +version. If the model changes under you, JevDK says so next to the calibration. + ## Comparing models: the results CSV JevDK doesn't tell you which model to use; your questions and your answers do. After a batch, diff --git a/examples/jevdk/Sources/JevDK/AppModel.swift b/examples/jevdk/Sources/JevDK/AppModel.swift index 5736ce9..b19a334 100644 --- a/examples/jevdk/Sources/JevDK/AppModel.swift +++ b/examples/jevdk/Sources/JevDK/AppModel.swift @@ -33,8 +33,12 @@ struct LocalChoice: Identifiable, Hashable { let repoID: String let sizeBytes: Int64? let isMoE: Bool + /// The exact version (Hugging Face commit) on this Mac; what an exported setup pins to. + var revision: String? = nil var displayName: String { repoID.split(separator: "/").last.map(String.init) ?? repoID } + /// The version as people read it: the commit's first 7 characters. + var shortRevision: String? { revision.map { String($0.prefix(7)) } } var sizeText: String { sizeBytes.map { ByteCountFormatter.string(fromByteCount: $0, countStyle: .file) } ?? "?" } } @@ -71,7 +75,9 @@ final class AppModel { /// Rows appended by the last write, for the status line. var lastAppend: (rows: Int, file: String)? var isRunning = false - var errorMessage: String? + var errorMessage: String? { didSet { if errorMessage != nil { notice = nil } } } + /// A confirmation to show after a save or export ("Exported 4 questions … pinned to 4dcb3d1"). + var notice: String? // Backends var useLocal = true { didSet { UserDefaults.standard.set(useLocal, forKey: "useLocal") } } @@ -119,7 +125,8 @@ final class AppModel { let probe = OpenJevDecisionProvider(mlx: mlx) models = mlx.installed.map { m in LocalChoice(repoID: m.repoID, sizeBytes: m.sizeBytes, - isMoE: probe.warnings(for: Self.openJevID(m.repoID)).contains(.mixtureOfExperts)) + isMoE: probe.warnings(for: Self.openJevID(m.repoID)).contains(.mixtureOfExperts), + revision: m.resolvedRevision) }.sorted { ($0.sizeBytes ?? 0) < ($1.sizeBytes ?? 0) } let verified = Set(models.map(\.repoID)) unverifiedRepos = ModelCatalog.cachedRepoIDs().filter { !verified.contains($0) } @@ -390,14 +397,73 @@ final class AppModel { panel.message = "A question set for your app: bundle it and load it with DecisionQuestionSet(contentsOf:)." guard panel.runModal() == .OK, let url = panel.url else { return } do { - let file = DecisionQuestionSet(name: set.name, questions: set.questions.map(\.sdkQuestion), tuning: exportTuning) + let tuning = exportTuning + let file = DecisionQuestionSet(name: set.name, questions: set.questions.map(\.sdkQuestion), tuning: tuning) try file.jsonData().write(to: url) errorMessage = nil + notice = "Exported \(set.questions.count) questions to \(url.lastPathComponent)" + Self.tuningSummary(tuning) + "." } catch { errorMessage = "Couldn't export: \(error.localizedDescription)" } } + /// ", for Qwen3-4B-4bit pinned to 4dcb3d1, calibrated" — what an export carries. + static func tuningSummary(_ t: DecisionQuestionSet.Tuning?) -> String { + guard let t, let model = t.model else { return " (no local model: questions only)" } + let name = model.rest.split(separator: "/").last.map(String.init) ?? model.rest + var s = ", for \(name)" + if let r = t.revision { s += " pinned to \(r.prefix(7))" } + s += t.calibration == nil ? ", not calibrated" : ", calibrated" + return s + } + + /// Add this setup (local model, its version, system instructions, calibration) to a jev-serve + /// config, creating one if needed (see the jev-serve example). An existing config keeps its other + /// models, listen address and token; the entry with this question set's name is replaced. + func exportServerConfig() { + guard let tuning = exportTuning else { + errorMessage = "Pick a local model first: jev-serve serves a local model." + return + } + let panel = NSSavePanel() + panel.allowedContentTypes = [.json] + panel.nameFieldStringValue = "jev-serve.json" + panel.message = "A jev-serve config. If the file exists, this setup is added to it (or replaces the one with the same name)." + guard panel.runModal() == .OK, let url = panel.url else { return } + var config: JevServeConfig + var created = false + if FileManager.default.fileExists(atPath: url.path) { + do { config = try JevServeConfig.read(url) } catch { + errorMessage = "\(url.lastPathComponent) isn't a jev-serve config, so it wasn't changed: \(error.localizedDescription)" + return + } + } else { + created = true + let alert = NSAlert() + alert.messageText = "Require a token?" + alert.informativeText = "With a token, jev-serve only answers requests that send it (Authorization: Bearer …). It's saved in the config file, which only you can read." + alert.addButton(withTitle: "Generate a Token") + alert.addButton(withTitle: "No Token") + config = JevServeConfig(token: alert.runModal() == .alertFirstButtonReturn ? JevServeConfig.newToken() : nil) + } + let name = Self.slug(set.name.isEmpty ? "decider" : set.name) + config.upsert(.init(name: name, tuning: tuning)) + do { + try config.write(to: url) + errorMessage = nil + notice = "\(created ? "Wrote" : "Updated") \(url.lastPathComponent): “\(name)”" + Self.tuningSummary(tuning) + + (config.token != nil ? ", token required" : "") + ". Serve it with: jev-serve --config \(url.lastPathComponent)" + } catch { + errorMessage = "Couldn't write \(url.lastPathComponent): \(error.localizedDescription)" + } + } + + /// "Customer support" → "customer-support", what requests send as "model". + static func slug(_ s: String) -> String { + let parts = s.lowercased().split { !($0.isLetter || $0.isNumber) } + return parts.isEmpty ? "decider" : parts.joined(separator: "-") + } + /// Load an answer set (CSV: `input`, then a column per question with the correct answer) into /// the batch list and the marks. Inputs with line breaks are joined onto one line, since the /// batch list is one input per line. diff --git a/examples/jevdk/Sources/JevDK/Calibration.swift b/examples/jevdk/Sources/JevDK/Calibration.swift index 578621d..d85b8f9 100644 --- a/examples/jevdk/Sources/JevDK/Calibration.swift +++ b/examples/jevdk/Sources/JevDK/Calibration.swift @@ -27,7 +27,9 @@ struct FittedCalibration: Codable, Hashable { func mismatch(repoID: String?, revision: String?, system: String, inputLabel: String) -> String? { var reasons: [String] = [] if repoID != self.repoID { reasons.append("it was fitted on \(self.repoID)") } - else if let revision, revision != self.revision { reasons.append("the model has been updated since") } + else if let revision, revision != self.revision { + reasons.append("the model has been updated since (fitted on version \(self.revision.prefix(7)), now \(revision.prefix(7)))") + } if system != self.system || inputLabel != self.inputLabel { reasons.append("the system instructions changed since") } return reasons.isEmpty ? nil : reasons.joined(separator: ", ") } diff --git a/examples/jevdk/Sources/JevDK/CalibrationSheet.swift b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift index 652da0e..b24286a 100644 --- a/examples/jevdk/Sources/JevDK/CalibrationSheet.swift +++ b/examples/jevdk/Sources/JevDK/CalibrationSheet.swift @@ -61,7 +61,7 @@ struct CalibrationSheet: View { .buttonStyle(.borderedProminent) .disabled(samples.isEmpty || model.selectedModel == nil) if let m = model.selectedModel { - Text("for \(m.repoID)").font(.jCaption).foregroundStyle(.secondary) + Text("for \(m.repoID)\(m.shortRevision.map { " @ \($0)" } ?? "")").font(.jCaption).foregroundStyle(.secondary) } } diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index 0e9287e..4ec263c 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -60,6 +60,8 @@ struct JevDKApp: App { } Section("For your app") { Button("Export Questions…") { model.exportForApp() }.keyboardShortcut("e") + Button("Export Server Config…") { model.exportServerConfig() } + .disabled(!model.useLocal || model.selectedModel == nil) } Section("Test data") { Button("Import Answers (CSV)…") { model.importAnswerSet() } @@ -162,7 +164,8 @@ struct ContentView: View { private var modelHelp: String { guard let m = model.selectedModel else { return "No SDK-verified models yet: open Models to choose one." } - var s = "\(m.repoID) · \(m.sizeText) · verified by the SDK" + var s = "\(m.repoID)\(m.shortRevision.map { " @ \($0)" } ?? "") · \(m.sizeText) · verified by the SDK" + if let r = m.revision { s += "\nVersion \(r): exports pin this exact version." } if m.isMoE { s += "\nMixture-of-experts: not recommended as a decider. Its probabilities can shift with how the prompt is split." } @@ -173,6 +176,9 @@ struct ContentView: View { HStack(spacing: 6) { if model.useLocal, let m = model.selectedModel { Text(m.repoID) + if let r = m.shortRevision { + Text("@ \(r)").foregroundStyle(.secondary).help("The model's version on this Mac (\(m.revision ?? r)). Exports pin it.") + } if m.isMoE { Text("· MoE: not recommended as a decider").foregroundStyle(.orange) } diff --git a/examples/jevdk/Sources/JevDK/ModelsSheet.swift b/examples/jevdk/Sources/JevDK/ModelsSheet.swift index bb9bd90..d561dc5 100644 --- a/examples/jevdk/Sources/JevDK/ModelsSheet.swift +++ b/examples/jevdk/Sources/JevDK/ModelsSheet.swift @@ -68,7 +68,7 @@ struct ModelsSheet: View { HStack { VStack(alignment: .leading) { Text(m.repoID).font(.system(size: 14, design: .monospaced)) - Text("\(m.sizeText)\(m.isMoE ? " · mixture-of-experts: not recommended as a decider" : "")") + Text("\(m.sizeText)\(m.shortRevision.map { " · version \($0)" } ?? "")\(m.isMoE ? " · mixture-of-experts: not recommended as a decider" : "")") .font(.jCaption).foregroundStyle(m.isMoE ? .orange : .secondary) } Spacer() diff --git a/examples/jevdk/Sources/JevDK/ResultsView.swift b/examples/jevdk/Sources/JevDK/ResultsView.swift index f76176c..ee9cf22 100644 --- a/examples/jevdk/Sources/JevDK/ResultsView.swift +++ b/examples/jevdk/Sources/JevDK/ResultsView.swift @@ -26,6 +26,19 @@ struct ResultsView: View { .frame(maxWidth: .infinity, alignment: .leading) .padding(.horizontal) .padding(.top, 8) + } else if let msg = model.notice { + HStack(alignment: .top) { + Label(msg, systemImage: "checkmark.circle.fill") + .foregroundStyle(.green) + .font(.jCallout) + .textSelection(.enabled) + Spacer() + Button { model.notice = nil } label: { Image(systemName: "xmark") } + .buttonStyle(.borderless) + .help("Dismiss") + } + .padding(.horizontal) + .padding(.top, 8) } switch mode { diff --git a/examples/jevdk/Sources/OpenJevKit/JevServeConfig.swift b/examples/jevdk/Sources/OpenJevKit/JevServeConfig.swift new file mode 100644 index 0000000..8f9e0c8 --- /dev/null +++ b/examples/jevdk/Sources/OpenJevKit/JevServeConfig.swift @@ -0,0 +1,87 @@ +import Foundation +import LocalLMLabSDKCore +import Security + +/// jev-serve's config file (the jev-serve example): where to listen, an optional token, and the +/// models to serve, each with the setup tested in JevDK. JevDK writes it (File → Export Server +/// Config…); jev-serve reads it. Each model's `tuning` is the SDK's `DecisionQuestionSet.Tuning`: +/// the model (`openjev:`), the revision it was tested on (jev-serve pins downloads to it), +/// the system instructions and the calibration. +public struct JevServeConfig: Codable, Hashable, Sendable { + public static let currentFormat = 1 + public static let defaultPort = 8746 + + public struct Listen: Codable, Hashable, Sendable { + public var host: String + public var port: Int + public init(host: String = "127.0.0.1", port: Int = JevServeConfig.defaultPort) { + self.host = host + self.port = port + } + } + + public struct Model: Codable, Hashable, Sendable { + /// What requests ask for in `"model"`, e.g. `support`. + public var name: String + public var tuning: DecisionQuestionSet.Tuning + public init(name: String, tuning: DecisionQuestionSet.Tuning) { + self.name = name + self.tuning = tuning + } + } + + public var format: Int + public var listen: Listen + /// When set, every request needs `Authorization: Bearer `. + public var token: String? + /// The model a request without `"model"` gets. + public var defaultModel: String? + public var maxStateCharacters: Int? + public var models: [Model] + + public init(listen: Listen = Listen(), token: String? = nil, defaultModel: String? = nil, + maxStateCharacters: Int? = 20_000, models: [Model] = []) { + self.format = Self.currentFormat + self.listen = listen + self.token = token + self.defaultModel = defaultModel + self.maxStateCharacters = maxStateCharacters + self.models = models + } + + /// Adds `model`, or replaces the entry with the same name, keeping the others; the first model + /// added becomes the default. + public mutating func upsert(_ model: Model) { + if let i = models.firstIndex(where: { $0.name == model.name }) { models[i] = model } else { models.append(model) } + if defaultModel == nil || !models.contains(where: { $0.name == defaultModel }) { defaultModel = models.first?.name } + } + + public static func read(_ url: URL) throws -> JevServeConfig { + let decoder = JSONDecoder() + decoder.dateDecodingStrategy = .iso8601 + let config = try decoder.decode(JevServeConfig.self, from: Data(contentsOf: url)) + guard config.format <= currentFormat else { + throw CocoaError(.fileReadCorruptFile, userInfo: [NSLocalizedDescriptionKey: + "This jev-serve config is format \(config.format); this version reads up to \(currentFormat)."]) + } + return config + } + + /// Writes the file readable and writable by this user only (it may hold the token). + public func write(to url: URL) throws { + let encoder = JSONEncoder() + encoder.outputFormatting = [.prettyPrinted, .sortedKeys, .withoutEscapingSlashes] + encoder.dateEncodingStrategy = .iso8601 + try encoder.encode(self).write(to: url, options: .atomic) + try FileManager.default.setAttributes([.posixPermissions: 0o600], ofItemAtPath: url.path) + } + + /// A random token for `Authorization: Bearer …` (32 bytes, base64url). + public static func newToken() -> String { + var bytes = [UInt8](repeating: 0, count: 32) + _ = SecRandomCopyBytes(kSecRandomDefault, bytes.count, &bytes) + return Data(bytes).base64EncodedString() + .replacingOccurrences(of: "+", with: "-").replacingOccurrences(of: "/", with: "_") + .replacingOccurrences(of: "=", with: "") + } +} From c78f7db17a4eb75946c09110146b348b79eca5bf Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 06:19:39 -0700 Subject: [PATCH 52/73] examples: add jev-serve, hosted Jev's API answered by OpenJev on this Mac MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit jev-serve takes OpenRouter's /api/alpha/decisions and Featherless's /v1/classifier requests and answers them with OpenJev on a local MLX model, from the config JevDK exports (File → Export Server Config…): each model pinned to its tested version (downloaded if missing), its system instructions and calibration, an optional token. Loopback by default; inputs never logged. JevServeKit is a library with tests that run without a GPU; --self-check verifies live. The SDK binaries come from ../jevdk's products. Needs an SDK release with JevWire. Docs: examples README, README, CHANGELOG, SDK guide §6c, the SDK skill, JevDK's README and guide. Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 4 + README.md | 3 + docs/sdk-guide.md | 20 ++ examples/README.md | 2 + examples/jev-serve/.gitignore | 2 + examples/jev-serve/Package.swift | 62 ++++ examples/jev-serve/README.md | 164 +++++++++++ .../jev-serve/Sources/JevServe/main.swift | 276 ++++++++++++++++++ .../Sources/JevServeKit/HTTPServer.swift | 203 +++++++++++++ .../Sources/JevServeKit/Service.swift | 229 +++++++++++++++ .../JevServeKitTests/JevServeKitTests.swift | 271 +++++++++++++++++ examples/jevdk/GUIDE.md | 10 + examples/jevdk/README.md | 3 + .../references/example-selector.md | 1 + 14 files changed, 1250 insertions(+) create mode 100644 examples/jev-serve/.gitignore create mode 100644 examples/jev-serve/Package.swift create mode 100644 examples/jev-serve/README.md create mode 100644 examples/jev-serve/Sources/JevServe/main.swift create mode 100644 examples/jev-serve/Sources/JevServeKit/HTTPServer.swift create mode 100644 examples/jev-serve/Sources/JevServeKit/Service.swift create mode 100644 examples/jev-serve/Tests/JevServeKitTests/JevServeKitTests.swift diff --git a/CHANGELOG.md b/CHANGELOG.md index 11e16e5..c46a420 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -214,6 +214,10 @@ API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdec ### Added — examples +- [`jev-serve`](examples/jev-serve/): hosted Jev's HTTP API (`/api/alpha/decisions`, + `/v1/classifier`) answered by OpenJev on this Mac, from a config JevDK exports (**File → Export + Server Config…**: the model pinned to its tested version, the system instructions, the + calibration, an optional token). Downloads the model if needed; loopback by default. - [`jevdk`](examples/jevdk/): JevDK, a playground for decision-model questions — run them on a local MLX model and on hosted Jev side by side, score a batch against your marked answers, calibrate, compare models in a results CSV, and export the tested questions for your app. With a diff --git a/README.md b/README.md index be25aa5..5417b08 100644 --- a/README.md +++ b/README.md @@ -176,6 +176,9 @@ Roughly simplest to fullest — every one runnable, with full annotated source i [Developer's guide](https://thisbrain.ai/locallm/jdk-guide.html); [download the app](https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/JevDK-0.1.0-arm64.dmg) (macOS 27, signed and notarized, a preview built on SDK 2.0.0-dev). +- **[examples/jev-serve/](examples/jev-serve/)** (2.0) — hosted Jev's HTTP API (OpenRouter's and + Featherless's) answered by OpenJev on your Mac: point existing hosted-Jev code, in any language, + at it to get private, on-device decisions, using the setup you tested and exported from JevDK. Plus **[examples/api-lab/](examples/api-lab/)** (scripts + a chat app for the OpenAI-compatible endpoint) and **[examples/localai-cli/](examples/localai-cli/)** / **[localai-cli-swift/](examples/localai-cli-swift/)** diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 73a9a2b..5aa94c5 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -2042,6 +2042,26 @@ let result = try await lab.evaluate(route: "support", questions: set.questions, when wrong (a big gap means a threshold can catch the misses), calibration error, and `samples` for `DecisionCalibration.fit`. +### Serving decisions over HTTP: `jev-serve` + +To use a local decider from another process or language, run the +[`jev-serve`](../examples/jev-serve/) example. It answers hosted Jev's API, OpenRouter's +`POST /api/alpha/decisions` and Featherless's `POST /v1/classifier`, with OpenJev on this Mac, so +existing hosted-Jev client code switches by changing its base URL. It serves the setup you tested +in JevDK (**File → Export Server Config…**): each model pinned to its tested revision (downloaded +if missing), with its wrapper and calibration; loopback by default, an optional bearer token. +Responses add `"fidelity": "tokenScored"` and `"calibrated"`, which `JevDecisionProvider` reads. + +The wire format is `JevWire` (Core), used by both the client (`JevDecisionProvider`) and the +server: + +```swift +let wire = try JevWire.decodeRequest(body) // model?, DecisionRequest (order kept) +try wire.request.validate(against: provider.limits) +let decision = try await provider.decide(wire.request, using: id) +let response = try JevWire.encodeResponse(decision, model: repo, request: wire.request) +``` + ### Decisions that need your product's knowledge A decider sees only `state` and the questions (there's no system prompt), so knowledge it needs, diff --git a/examples/README.md b/examples/README.md index 062db36..5831813 100644 --- a/examples/README.md +++ b/examples/README.md @@ -23,6 +23,7 @@ Code samples for LocalLM Lab, split by which feature they use. | [vistanova/](vistanova/) | LocalLM Lab SDK (Core **+ Inference**) | A tiny local search engine: a SwiftUI app that runs web search through a Tavily MCP server on one local model and summarizes on another — by default a downloaded MLX model **shipped pinned to an exact commit** (`MLXModelProvider(pinnedRevisions:)`), so a new upstream version is never picked up silently. Also shows a hand-written `Tool` over an MCP tool, and the defenses a small model needs: verify from `session.events` that it really called the tool, probe `capabilityProbe` before trusting structured output. | | [aiql/](aiql/) | LocalLM Lab SDK (Core **+ Inference**) | "Ask your data": a SwiftUI app that pulls an MCP-fronted dataset and writes the spreadsheet you asked for — `FileBackedTool` + the data verbs (`jsonToCsv` / `filterRows` / `sortRows` / …, guide §8b) do every row-level step so a local MLX model can't fabricate a value. Sandboxed; MCP OAuth via the `aiql://` URL scheme. | | [jevdk/](jevdk/) | LocalLM Lab SDK (Core **+ Inference + Remote**), **2.0** | The decision API (Jev): a playground for the yes/no, choice and score questions an app asks a decision model. Runs them on a local MLX model (`OpenJevDecisionProvider`) and on hosted Jev (TypeSafe via OpenRouter, Featherless) side by side, scores a batch against your marked answers, fits calibration, appends runs to a results CSV for comparing models, and exports the tested questions for your app (`DecisionQuestionSet`). Comes with [a developer's guide](jevdk/GUIDE.md). | +| [jev-serve/](jev-serve/) | LocalLM Lab SDK (Core **+ Inference**), **2.0** | Hosted Jev's HTTP API (OpenRouter's `/api/alpha/decisions`, Featherless's `/v1/classifier`) answered by OpenJev on a local MLX model: code that calls hosted Jev, in any language, switches to private on-device decisions by changing its base URL. Serves the setup tested in JevDK (model pinned to its version, system instructions, calibration) from JevDK's exported config; downloads the model if needed; loopback, optional token. | | [mcp-chat/](mcp-chat/) | LocalLM Lab SDK (Core **+ Inference** + `Components` + `MCPAppsHost`), **2.0** | MCP Apps in a chat: a local MLX model (or Apple's on-device model) whose tool calls show the server's interactive widget inline in the conversation — Todoist's task list, a system monitor. The widget's own calls go through the session, so they meet the same per-server trust / approval as the model's. Streaming replies, reasoning split out, conversations that survive relaunch. Sandboxed `.app`. | **Path A vs Path B**, for the four Core-based examples above: two ways to turn a connector or MCP @@ -73,6 +74,7 @@ build against the latest stable release. No environment variable is needed for e | `components-updates-demo` | SwiftUI, no system permissions, simulated data | ✅ via `Package.swift` / `swift run` (no `.xcodeproj`) | — (no `packaging/`) | | `workspace-buddy`, `workspace-buddy-local`, `aiql` | SwiftUI + App Sandbox | ✅ via the committed `.xcodeproj` — **Automatic** signing (as above); a stable team identity is what lets the security-scoped bookmark survive a rebuild. Bare `swift run` is compile-only. | same as `plate-today` | | `jevdk` | SwiftUI, no system permissions, no sandbox; hosted deciders optional (Featherless demo needs no key) | ✅ via the committed `.xcodeproj` — ad-hoc signed (*Sign to Run Locally*), not sandboxed, no account needed; or `swift build -c release` and run `.build/release/JevDK` | — (no `packaging/`) | +| `jev-serve` | CLI (an HTTP server on this Mac) | ✅ `swift build -c release`, then `.build/release/jev-serve --config jev-serve.json` (from JevDK) or `--model ` | — (no `packaging/`) | | `mcp-chat` | SwiftUI + App Sandbox (network) | Compile-only via `Package.swift` / `swift build` (no `.xcodeproj`) | **any** identity, or **none** (`APP_IDENTITY=-`) — `packaging/build-and-sign.sh` | "Apple Development" = the free identity Xcode creates once you add any Apple ID under diff --git a/examples/jev-serve/.gitignore b/examples/jev-serve/.gitignore new file mode 100644 index 0000000..5104546 --- /dev/null +++ b/examples/jev-serve/.gitignore @@ -0,0 +1,2 @@ +.build/ +jev-serve.json diff --git a/examples/jev-serve/Package.swift b/examples/jev-serve/Package.swift new file mode 100644 index 0000000..342bcbd --- /dev/null +++ b/examples/jev-serve/Package.swift @@ -0,0 +1,62 @@ +// swift-tools-version: 6.4 +import PackageDescription + +// jev-serve — answers hosted Jev's HTTP API (OpenRouter's /api/alpha/decisions, Featherless's +// /v1/classifier) with OpenJev on a local MLX model. Code that calls hosted Jev switches to +// private, on-device decisions by changing its base URL. See README.md. +// +// Built on the SDK: OpenJevDecisionProvider and MLXModelProvider (Inference) for decisions and +// model downloads, JevWire (Core) for the wire format. The config file is JevDK's (File → Export +// Server Config…), read with OpenJevKit's JevServeConfig. JevServeKit (the server and routes) is a +// library so it can be tested without a GPU; the tests call it with the SDK's own hosted-Jev +// client (JevDecisionProvider, Remote). +// +// The SDK binaries (LocalLMLabSDKCore, LocalLMLabSDKInference, LocalLMLabSDKRemote) come from +// ../jevdk, which declares them as binaryTargets on a GitHub Release and vends them as products. +// This manifest must NOT declare its own: two packages declaring a binary target of the same name +// is a hard SwiftPM error. So jev-serve builds against whichever SDK version JevDK's +// `defaultSDKVersion` / LOCALLM_SDK_VERSION selects (2.0 releases only). +// +// Requires SDK 2.0.0 or later, macOS 27 + Xcode 27. No Metal Toolchain needed — the prebuilt +// Inference xcframework bundles the compiled shaders. +let package = Package( + name: "JevServe", + platforms: [.macOS("27.0")], + dependencies: [ + .package(path: "../jevdk"), + ], + targets: [ + .target( + name: "JevServeKit", + dependencies: [ + .product(name: "LocalLMLabSDKCore", package: "jevdk"), + .product(name: "LocalLMLabSDKInference", package: "jevdk"), + ], + swiftSettings: [.swiftLanguageMode(.v5)] + ), + .executableTarget( + name: "jev-serve", + dependencies: [ + "JevServeKit", + .product(name: "LocalLMLabSDKCore", package: "jevdk"), + .product(name: "LocalLMLabSDKInference", package: "jevdk"), + .product(name: "OpenJevKit", package: "jevdk"), + ], + path: "Sources/JevServe", + swiftSettings: [.swiftLanguageMode(.v5)], + linkerSettings: [ + // Bare SwiftPM executables get no LC_RPATH; SwiftPM extracts the xcframework + // slices next to the built binary. Same fix as jevdk / code-buddy. + .unsafeFlags(["-Xlinker", "-rpath", "-Xlinker", "@executable_path"]) + ] + ), + .testTarget( + name: "JevServeKitTests", + dependencies: [ + "JevServeKit", + .product(name: "LocalLMLabSDKCore", package: "jevdk"), + .product(name: "LocalLMLabSDKRemote", package: "jevdk"), + ] + ), + ] +) diff --git a/examples/jev-serve/README.md b/examples/jev-serve/README.md new file mode 100644 index 0000000..af8856f --- /dev/null +++ b/examples/jev-serve/README.md @@ -0,0 +1,164 @@ +# jev-serve + +Hosted Jev's HTTP API, answered on your Mac. `jev-serve` takes the same requests as TypeSafe's Jev +on OpenRouter (`POST /api/alpha/decisions`) and Featherless's Simple Jev (`POST /v1/classifier`) +and answers them with **OpenJev**, the LocalLM Lab SDK's decider, on a local MLX model. Code that +already calls hosted Jev, in any language, switches to private, free, on-device decisions by +changing its base URL. + +New to decision models? Start with [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html) +and the [JevDK guide](https://thisbrain.ai/locallm/jdk-guide.html). + +```bash +swift build -c release +.build/release/jev-serve --config jev-serve.json +``` + +Needs an Apple-silicon Mac, macOS 27 and Xcode 27 (Swift 6.4). + +## 1. Make a config in JevDK + +Design and test your questions in [JevDK](../jevdk/), pick and calibrate a local model, then +**File → Export Server Config…**. That writes `jev-serve.json`: where to listen, an optional +token, and your tested setup (the model, its exact version, the system instructions and the +calibration). Export again from another question set to add a second entry to the same file. + +```json +{ + "format": 1, + "listen": { "host": "127.0.0.1", "port": 8746 }, + "token": "…", + "defaultModel": "customer-support", + "maxStateCharacters": 20000, + "models": [ + { "name": "customer-support", + "tuning": { + "model": "openjev:mlx-community/Qwen3-4B-4bit", + "revision": "4dcb3d101c2a062e5c1d4bb173588c54ea6c4d25", + "system": "You answer questions about a piece of text. …", + "inputLabel": "Text", + "calibration": { "noulTemperature": 9.11, "choiceTemperature": 0.25, "scoreTemperature": 1 } } } + ] +} +``` + +No config yet? `jev-serve --model mlx-community/Qwen3-4B-4bit` serves one model with the default +system instructions and no calibration. + +## 2. Start it + +``` +$ jev-serve --config jev-serve.json +jev-serve on http://127.0.0.1:8746 (token required) + customer-support (default) mlx-community/Qwen3-4B-4bit @ 4dcb3d1 calibrated +OpenRouter clients: base URL http://127.0.0.1:8746/api (POST /alpha/decisions) +Featherless clients: base URL http://127.0.0.1:8746/v1 (POST /classifier) +``` + +**The model is made ready first.** Each model is pinned to the version in the config (the one you +tested and calibrated). If that version is on this Mac, it's used. If another tool left a copy in +the Hugging Face cache, it's verified and only missing files are fetched. If it isn't here, it's +checked (format, architecture, memory, disk) and downloaded, with progress. Then the default model +is loaded, and only then does jev-serve accept requests. `--no-download` makes a missing model an +error instead. + +## 3. Call it + +The request and response are hosted Jev's. With `curl`: + +```bash +curl -s http://127.0.0.1:8746/v1/classifier \ + -H "Authorization: Bearer $JEV_SERVE_TOKEN" -H "Content-Type: application/json" \ + -d '{ + "model": "customer-support", + "state": "Nothing loads since this morning, my whole team is stuck.", + "questions": { + "team": {"type": "choice", "instructions": "Which team should handle this message?", + "criteria": {"billing": "payments, invoices and refunds", + "technical": "bugs, outages and product errors", + "account": "profile, login and subscription changes"}}, + "refund": {"type": "noul", "instructions": "Does the customer explicitly ask for money back?"}, + "urgency": {"type": "score", "instructions": "How urgent is it?", "criteria": ["can wait", "soon", "now"]} + } + }' +``` + +```json +{"model": "mlx-community/Qwen3-4B-4bit", + "answers": { + "team": {"type": "choice", "choice": "technical", "confidence": 0.99, "probabilities": {"billing": 0.0, "technical": 0.99, "account": 0.01}}, + "refund": {"type": "noul", "noul": 0.03}, + "urgency": {"type": "score", "score": 1.44, "confidence": 0.56, "probabilities": {"0": 0.0, "1": 0.56, "2": 0.44}, + "legend": {"0": "can wait", "1": "soon", "2": "now"}}}, + "usage": {"input_tokens": 56, "output_tokens": 0, "cost": 0}, + "fidelity": "tokenScored", "calibrated": true} +``` + +- **`model`**: a model's name from the config, or its Hugging Face repo id. Leave it out for the + config's default. +- **`state`**: text, or any JSON object. +- From an existing hosted-Jev client, change only the base URL (and the token). From Swift, the + SDK's own `JevDecisionProvider` works too: point a `JevProviderConfig` at + `http://127.0.0.1:8746/v1/classifier`. + +| Route | | +|---|---| +| `POST /api/alpha/decisions` | OpenRouter / TypeSafe Jev's path | +| `POST /v1/classifier` | Featherless's path (same body) | +| `GET /v1/models` | The served models, with version and whether calibrated | +| `GET /health` | `{"status":"ok"}`; needs no token | + +Errors use hosted Jev's shape, `{"error": {"message", "type", "code"}}`: 400 for a request that +can't be asked (with what's wrong), 401 without the token, 404 for an unknown model or route, 413 +for a body over 1 MB, 503 when too many requests are waiting. + +## How it differs from TypeSafe's Jev + +| | TypeSafe Jev | jev-serve (OpenJev) | +|---|---|---| +| Probabilities | From a model trained to decide | Read off a language model's next-token probabilities (`"fidelity": "tokenScored"`); honest only once calibrated (`"calibrated": true`) | +| Limits | 255 choice options, 50 score levels | 64 questions, 26 choice options, 10 score levels | +| `confidence` | Reported separately | The top answer's probability | +| `noul` | — | Not clamped (Featherless clamps to 0.01–0.99) | +| `usage` | Tokens and cost | The input's tokens; nothing generated; cost 0 | +| `messages`, images, `options` | Featherless / OpenRouter extras | Ignored | + +## Security + +- **This Mac only by default** (`127.0.0.1`). Any app of yours can call it; the **token** keeps out + everything that doesn't have it. The config file is readable only by you. +- **Plain HTTP.** Listening on another address (`"host": "0.0.0.0"`, `--host`) works but logs a + warning: inputs and the token would cross the network unencrypted. +- **Inputs aren't logged.** One line per request (route, model, number of questions, status, time). + `--log-bodies` logs the bodies, for debugging only. +- No CORS headers: a web page in a browser can't call it. + +## Options + +``` +jev-serve --config | --model + [--host 127.0.0.1] [--port 8746] [--no-download] [--log-bodies] [--self-check] +``` + +`--self-check` starts on a spare port, asks every served model the same questions over HTTP and +directly, checks the answers match, and exits (0 if they all do). Run it after changing the +config, the model or the SDK. + +## Testing + +```bash +swift test # routes, token, limits, queue, HTTP, SDK client round trip +.build/release/jev-serve --config jev-serve.json --self-check # the same, live on your model +``` + +The unit tests run the real server and routes with a scripted decider (no GPU needed), and call it +with the SDK's own hosted-Jev client, `JevDecisionProvider`, so the wire format is checked both ways. + +## How it works + +About 500 lines on the SDK's public API, in `JevServeKit` (the server and routes, testable) +and the `jev-serve` command: `JevWire` (Core) reads and writes hosted Jev's JSON (the +same code the SDK's `JevDecisionProvider` uses as a client), `OpenJevDecisionProvider` and +`MLXModelProvider` (Inference) answer and download, and JevDK's `JevServeConfig` reads the config. +The HTTP server is `Network.framework`, no other dependency. Decisions run one at a time (one GPU) +in arrival order, about 80–150 ms each on Qwen3-4B. diff --git a/examples/jev-serve/Sources/JevServe/main.swift b/examples/jev-serve/Sources/JevServe/main.swift new file mode 100644 index 0000000..91dc56e --- /dev/null +++ b/examples/jev-serve/Sources/JevServe/main.swift @@ -0,0 +1,276 @@ +// jev-serve — hosted Jev's HTTP API, answered by OpenJev on this Mac. +// +// jev-serve --config jev-serve.json # the config JevDK writes (File → Export Server Config…) +// jev-serve --model mlx-community/Qwen3-4B-4bit # quick try: default wrapper, no calibration +// +// Options: --host 127.0.0.1 --port 8746 --no-download --log-bodies --self-check --help +// +// At startup each configured model is made ready (found, verified or downloaded, pinned to the +// version in the config), the default one is loaded, and only then does it start listening. + +import Foundation +import LocalLMLabSDKCore +import JevServeKit +import LocalLMLabSDKInference +import OpenJevKit + +let usage = """ + jev-serve — hosted Jev's API (OpenRouter /api/alpha/decisions, Featherless /v1/classifier), on this Mac. + + Usage: + jev-serve --config serve the models in a config written by JevDK (File → Export Server Config…) + jev-serve --model serve one Hugging Face MLX model, default system instructions, no calibration + + Options: + --host
listen address (default from the config, else 127.0.0.1) + --port port (default from the config, else 8746) + --no-download fail instead of downloading a missing model + --log-bodies also log request and response bodies (they hold your inputs) + --self-check start on a spare port, ask every model the same questions over HTTP and directly, + check the answers match, and exit (0 = all match) + --help + """ + +func fail(_ message: String) -> Never { + FileHandle.standardError.write(Data("jev-serve: \(message)\n".utf8)) + exit(1) +} + +// MARK: Options + +var args = Array(CommandLine.arguments.dropFirst()) +func option(_ name: String) -> String? { + guard let i = args.firstIndex(of: name) else { return nil } + guard i + 1 < args.count else { fail("\(name) needs a value") } + let v = args[i + 1] + args.removeSubrange(i...(i + 1)) + return v +} +func flag(_ name: String) -> Bool { + guard let i = args.firstIndex(of: name) else { return false } + args.remove(at: i) + return true +} +setvbuf(stdout, nil, _IOLBF, 0) // line-buffered, so the banner shows when piped or logged +if flag("--help") || flag("-h") { print(usage); exit(0) } +let configPath = option("--config") +let modelOption = option("--model") +let hostOption = option("--host") +let portOption = option("--port") +let noDownload = flag("--no-download") +let logBodies = flag("--log-bodies") +let selfCheck = flag("--self-check") +if let extra = args.first { fail("unknown option \(extra)\n\n\(usage)") } + +// MARK: Config + +var config: JevServeConfig +if let configPath { + do { config = try JevServeConfig.read(URL(fileURLWithPath: (configPath as NSString).expandingTildeInPath)) } catch { + fail("can't read \(configPath): \(error.localizedDescription)") + } + if let modelOption { fail("use --config or --model, not both (got --model \(modelOption))") } +} else if let modelOption { + let name = modelOption.split(separator: "/").last.map { $0.lowercased() } ?? modelOption + config = JevServeConfig(models: [.init(name: name, tuning: .init(model: ModelID(scheme: "openjev", rest: modelOption)))]) + config.upsert(config.models[0]) +} else { + print(usage) + exit(1) +} +if let hostOption { config.listen.host = hostOption } +if let portOption { + guard let p = Int(portOption), (1...65_535).contains(p) else { fail("--port must be 1–65535") } + config.listen.port = p +} +guard !config.models.isEmpty else { fail("the config has no models") } +var repos: [String] = [] +var pins: [String: String] = [:] +for m in config.models { + guard let id = m.tuning.model, !id.rest.isEmpty else { fail("model “\(m.name)” has no model id (tuning.model)") } + if !repos.contains(id.rest) { repos.append(id.rest) } + if let r = m.tuning.revision { + if let other = pins[id.rest], other != r { + fail("\(id.rest) is pinned to two versions (\(other.prefix(7)), \(r.prefix(7))); one model can only be served at one version") + } + pins[id.rest] = r + } +} + +// MARK: Models: found, verified or downloaded, pinned + +// Pinned to the versions in the config, so a download fetches exactly what was tested and +// calibrated. A model with no version is pinned on first download (trust on first use). +let mlx = MLXModelProvider(residentModelLimit: max(1, repos.count), pinnedRevisions: pins, pinStore: MLXFilePinStore()) + +@MainActor func ensureReady(_ repo: String) async { + let id = ModelID(scheme: "mlx", rest: repo)! + switch mlx.availability(for: id) { + case .available: + return + case .unavailable(_, let detail): + fail("\(repo) can't run here: \(detail)") + case .needsCredential: + fail("\(repo) needs a Hugging Face credential") + case .notDownloaded: + break + @unknown default: + break + } + let other = mlx.installed.first { $0.repoID == repo }?.resolvedRevision + let what: String + if let want = pins[repo], let other, other != want { + what = "\(repo) version \(want.prefix(7)) (the one in the config) isn't on this Mac; version \(other.prefix(7)) is" + } else { + what = "\(repo) isn't on this Mac" + } + if noDownload { fail("\(what), and --no-download is set. Drop --no-download to fetch it, or download it in JevDK's Models.") } + if other != nil { jevServeLog("\(what): fetching the configured version") } + if let pre = try? await mlx.validate(repo), !pre.passed { + fail("pre-flight failed for \(repo) (\(pre.failedStage?.rawValue ?? "?")): \(pre.detail ?? "")") + } + let version = pins[repo].map { " @ \($0.prefix(7))" } ?? "" + jevServeLog("downloading \(repo)\(version) (a copy already in the Hugging Face cache is verified, only missing files fetched)…") + do { + var last = -1 + for try await event in mlx.download(repo) { + if case .progress(_, _, let f) = event { + let pct = Int(f * 100) + if pct != last { + last = pct + FileHandle.standardError.write(Data("\u{1B}[2K\r \(repo)\(version) \(pct)%".utf8)) + } + } + } + FileHandle.standardError.write(Data("\u{1B}[2K\r".utf8)) + jevServeLog("\(repo): ready") + } catch { + FileHandle.standardError.write(Data("\n".utf8)) + fail("download of \(repo) failed: \(error.localizedDescription)") + } +} + +for repo in repos { await ensureReady(repo) } + +// MARK: Providers + +var served: [ServedModel] = [] +for m in config.models { + let repo = m.tuning.model!.rest + var tuning = m.tuning + let set = DecisionQuestionSet(name: m.name, questions: [], tuning: tuning) + let probe = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) + let id = ModelID(scheme: probe.scheme, rest: repo)! + if let why = probe.tuningMismatch(for: id) { + jevServeLog("warning: “\(m.name)”: \(why); serving it without its calibration") + tuning.calibration = nil + } + if probe.warnings(for: id).contains(.mixtureOfExperts) { + jevServeLog("warning: \(repo) is a mixture-of-experts model; its probabilities are unstable as a decider") + } + let provider = OpenJevDecisionProvider(mlx: mlx, tunedWith: DecisionQuestionSet(name: m.name, questions: [], tuning: tuning)) + let revision = mlx.effectivePin(for: repo)?.revision ?? tuning.revision + ?? mlx.installed.first { $0.repoID == repo }?.resolvedRevision + served.append(ServedModel(name: m.name, repo: repo, revision: revision, decider: OpenJevDecider(provider: provider, id: id))) +} +let defaultName = config.defaultModel ?? served[0].name +guard let defaultModel = served.first(where: { $0.name == defaultName }) else { + fail("defaultModel “\(defaultName)” isn't one of the models") +} +jevServeLog("loading \(defaultModel.repo)…") +if let d = defaultModel.decider as? OpenJevDecider { await d.provider.prewarm(d.id) } + +// MARK: Serve + +let service = Service(models: served, defaultModel: defaultName, token: config.token, + maxStateCharacters: config.maxStateCharacters, queueLimit: 32, logBodies: logBodies) +let server: HTTPServer +do { + server = try HTTPServer(host: config.listen.host, port: selfCheck ? 0 : UInt16(config.listen.port), maxBodyBytes: 1_000_000) { req in + await service.handle(req) + } + try await server.start() +} catch { + fail("\(error.localizedDescription) (\(config.listen.host):\(config.listen.port); is another jev-serve running? try --port)") +} + +if selfCheck { exit(await runSelfCheck(port: server.port ?? 0)) } + +let base = "http://\(config.listen.host):\(config.listen.port)" +print("jev-serve on \(base)\(config.token != nil ? " (token required)" : "")") +for m in served { + let state = m.decider.calibrated ? "calibrated" : "not calibrated" + print(" \(m.name)\(m.name == defaultName ? " (default)" : "") \(m.repo)\(m.revision.map { " @ \($0.prefix(7))" } ?? "") \(state)") +} +print("OpenRouter clients: base URL \(base)/api (POST /alpha/decisions)") +print("Featherless clients: base URL \(base)/v1 (POST /classifier)") +if !["127.0.0.1", "localhost", "::1"].contains(config.listen.host) { + jevServeLog("warning: listening on \(config.listen.host): inputs\(config.token != nil ? " and the token" : "") cross the network unencrypted (plain HTTP)") +} +if logBodies { jevServeLog("warning: --log-bodies is on: request and response bodies (your inputs) are logged") } + +while true { try await Task.sleep(for: .seconds(3600)) } + +// MARK: Self-check + +/// Asks every served model the same questions through the running server (HTTP + the wire format) +/// and directly (its decider), and checks the answers match. The live counterpart of the unit tests, +/// which can't load MLX. +@MainActor func runSelfCheck(port: UInt16) async -> Int32 { + let questions: [DecisionQuestion] = [ + .choice("team", "Which team should handle this customer message?", criteria: [ + "billing": "payments, invoices and refunds", "technical": "bugs, outages and product errors", + "account": "profile, login and subscription changes"]), + .noul("refund", "Does the customer explicitly ask for money back?"), + .score("urgency", "How urgent is it?", criteria: ["can wait", "soon", "now"]), + ] + let inputs = ["I see two charges for my Pro plan this month. Please refund one.", + "Nothing loads since this morning, my whole team is stuck."] + var ok = true + for m in served { + for input in inputs { + let request = DecisionRequest(state: .text(input), questions: questions) + do { + var http = URLRequest(url: URL(string: "http://127.0.0.1:\(port)/v1/classifier")!) + http.httpMethod = "POST" + http.setValue("application/json", forHTTPHeaderField: "Content-Type") + if let t = config.token { http.setValue("Bearer \(t)", forHTTPHeaderField: "Authorization") } + http.httpBody = try JevWire.encodeRequest(request, model: m.name) + let (data, response) = try await URLSession.shared.data(for: http) + guard (response as? HTTPURLResponse)?.statusCode == 200 else { + print("FAIL \(m.name): HTTP \((response as? HTTPURLResponse)?.statusCode ?? 0): \(JevWire.errorMessage(in: data) ?? "")") + ok = false; continue + } + let served = try JevWire.decodeResponse(data, request: request) + let direct = try await m.decider.decide(request).decision + let diff = maxDifference(served.answers, direct.answers) + let same = diff < 1e-9 && served.fidelity == direct.fidelity + ok = ok && same + print("\(same ? "ok " : "FAIL") \(m.name) \(input.prefix(36))… max difference \(diff) \(served.fidelity.map { "\($0)" } ?? "no fidelity")") + } catch { + print("FAIL \(m.name): \(error.localizedDescription)") + ok = false + } + } + } + print(ok ? "self-check passed" : "self-check FAILED") + return ok ? 0 : 1 +} + +func maxDifference(_ a: [String: DecisionAnswer], _ b: [String: DecisionAnswer]) -> Double { + func flat(_ x: [String: DecisionAnswer]) -> [String: Double] { + var out: [String: Double] = [:] + for (id, ans) in x { + switch ans { + case .noul(let p): out[id] = p + case .choice(let c): out["\(id).\(c.choice)"] = 1; for (k, p) in c.probabilities { out["\(id).p.\(k)"] = p } + case .score(let sc): out["\(id).score"] = sc.score; for (i, p) in sc.probabilities.enumerated() { out["\(id).p\(i)"] = p } + @unknown default: break + } + } + return out + } + let x = flat(a), y = flat(b) + guard Set(x.keys) == Set(y.keys) else { return .infinity } + return x.map { abs($0.value - y[$0.key]!) }.max() ?? 0 +} diff --git a/examples/jev-serve/Sources/JevServeKit/HTTPServer.swift b/examples/jev-serve/Sources/JevServeKit/HTTPServer.swift new file mode 100644 index 0000000..168cb35 --- /dev/null +++ b/examples/jev-serve/Sources/JevServeKit/HTTPServer.swift @@ -0,0 +1,203 @@ +import Foundation +import LocalLMLabSDKCore +import Network + +/// A request, as much of HTTP/1.1 as jev-serve needs: method, path (query dropped), headers +/// (names lowercased), and a `Content-Length` body. +public struct HTTPRequest: Sendable { + public var method: String + public var path: String + public var headers: [String: String] + public var body: Data + + public init(method: String, path: String, headers: [String: String] = [:], body: Data = Data()) { + self.method = method + self.path = path + self.headers = headers + self.body = body + } +} + +public struct HTTPResponse: Sendable { + public var status: Int + public var body: Data + public var contentType = "application/json" + + public static func json(_ status: Int, _ body: Data) -> HTTPResponse { HTTPResponse(status: status, body: body) } +} + +/// A minimal HTTP/1.1 server on `Network.framework`: keep-alive, `Content-Length` bodies only (no +/// chunked uploads), a size cap on headers and body, and an idle timeout. Enough for JSON APIs on +/// this Mac; not a general web server. +public final class HTTPServer: @unchecked Sendable { + public typealias Handler = @Sendable (HTTPRequest) async -> HTTPResponse + + private let listener: NWListener + private let handler: Handler + private let maxBodyBytes: Int + private let maxHeaderBytes = 16 * 1024 + private let idleTimeout: TimeInterval = 30 + private let queue = DispatchQueue(label: "jev-serve.http") + + /// `port` 0 lets the system pick one; read it from `port` after `start()`. + public init(host: String, port: UInt16, maxBodyBytes: Int, handler: @escaping Handler) throws { + let nwPort: NWEndpoint.Port + if port == 0 { nwPort = .any } else { + guard let p = NWEndpoint.Port(rawValue: port) else { throw ServerError("invalid port \(port)") } + nwPort = p + } + let params = NWParameters.tcp + params.allowLocalEndpointReuse = true + params.requiredLocalEndpoint = .hostPort(host: NWEndpoint.Host(host), port: nwPort) + self.listener = try NWListener(using: params) + self.handler = handler + self.maxBodyBytes = maxBodyBytes + } + + /// The bound port, once started. + public var port: UInt16? { listener.port?.rawValue } + + /// Stops accepting connections; open ones finish or time out. + public func stop() { listener.cancel() } + + /// Starts listening; returns once the port is bound, or throws (e.g. the port is in use). + public func start() async throws { + try await withCheckedThrowingContinuation { (cont: CheckedContinuation) in + // State updates arrive on `queue` (serial), so this flag is only touched there. + final class Once: @unchecked Sendable { var done = false } + let once = Once() + listener.stateUpdateHandler = { state in + guard !once.done else { + if case .failed(let e) = state { FileHandle.standardError.write(Data("listener failed: \(e)\n".utf8)) } + return + } + switch state { + case .ready: once.done = true; cont.resume() + case .failed(let e): once.done = true; cont.resume(throwing: ServerError("can't listen: \(e)")) + case .cancelled: once.done = true; cont.resume(throwing: ServerError("listener cancelled")) + default: break + } + } + listener.newConnectionHandler = { [weak self] conn in self?.accept(conn) } + listener.start(queue: queue) + } + } + + private func accept(_ conn: NWConnection) { + let c = Connection(conn: conn, server: self) + conn.start(queue: queue) + c.readMore() + } + + // MARK: Per connection + + private final class Connection: @unchecked Sendable { + let conn: NWConnection + let server: HTTPServer // strong: callbacks can outlive the caller's reference (no cycle: the server doesn't keep connections) + var buffer = Data() + var idle: DispatchWorkItem? + + init(conn: NWConnection, server: HTTPServer) { + self.conn = conn + self.server = server + } + + func resetIdle() { + idle?.cancel() + let item = DispatchWorkItem { [weak self] in self?.conn.cancel() } + idle = item + server.queue.asyncAfter(deadline: .now() + server.idleTimeout, execute: item) + } + + func readMore() { + resetIdle() + conn.receive(minimumIncompleteLength: 1, maximumLength: 64 * 1024) { [self] data, _, isComplete, error in + if let data { buffer.append(data) } + if error != nil { close(); return } + if process() { return } + if isComplete { close(); return } + readMore() + } + } + + /// Handles a complete request in the buffer, if there is one. Returns true when it took + /// over the connection (responding, then reading again or closing). + func process() -> Bool { + guard let headerEnd = buffer.range(of: Data("\r\n\r\n".utf8)) else { + if buffer.count > server.maxHeaderBytes { fail(431, "Request headers too large."); return true } + return false + } + let head = String(decoding: buffer[buffer.startIndex..= 0 else { fail(400, "Invalid Content-Length."); return true } + guard length <= server.maxBodyBytes else { fail(413, "The body is over \(server.maxBodyBytes) bytes."); return true } + let bodyStart = headerEnd.upperBound + guard buffer.count - (bodyStart - buffer.startIndex) >= length else { return false } // wait for the rest + let body = buffer.subdata(in: bodyStart..<(bodyStart + length)) + buffer.removeSubrange(buffer.startIndex..<(bodyStart + length)) + var path = String(requestLine[1]) + if let q = path.firstIndex(of: "?") { path = String(path[.. String { + switch s { + case 200: "OK" + case 400: "Bad Request" + case 401: "Unauthorized" + case 404: "Not Found" + case 405: "Method Not Allowed" + case 411: "Length Required" + case 413: "Payload Too Large" + case 431: "Request Header Fields Too Large" + case 500: "Internal Server Error" + case 503: "Service Unavailable" + default: "Status" + } + } + } +} + +public struct ServerError: LocalizedError { + public let message: String + public init(_ message: String) { self.message = message } + public var errorDescription: String? { message } +} diff --git a/examples/jev-serve/Sources/JevServeKit/Service.swift b/examples/jev-serve/Sources/JevServeKit/Service.swift new file mode 100644 index 0000000..4d88d16 --- /dev/null +++ b/examples/jev-serve/Sources/JevServeKit/Service.swift @@ -0,0 +1,229 @@ +import Foundation +import LocalLMLabSDKCore +import LocalLMLabSDKInference + +/// What answers a served model's questions. jev-serve uses `OpenJevDecider` (OpenJev on MLX); tests +/// use a scripted one, so routing, auth, limits and the queue are tested without a GPU. +public protocol Decider: Sendable { + var limits: DecisionLimits { get } + var calibrated: Bool { get } + /// The decision, and the input tokens it read (for `usage`). + func decide(_ request: DecisionRequest) async throws -> (decision: Decision, inputTokens: Int?) +} + +/// OpenJev on a local MLX model, with one config entry's wrapper and calibration. +public struct OpenJevDecider: Decider { + public let provider: OpenJevDecisionProvider + public let id: ModelID + + public init(provider: OpenJevDecisionProvider, id: ModelID) { + self.provider = provider + self.id = id + } + + public var limits: DecisionLimits { provider.limits } + public var calibrated: Bool { provider.calibration != nil } + + public func decide(_ request: DecisionRequest) async throws -> (decision: Decision, inputTokens: Int?) { + let (decision, diagnostics) = try await provider.decideWithDiagnostics(request, using: id) + return (decision, diagnostics.sharedPrefixTokens) + } +} + +/// One model jev-serve answers for. +public struct ServedModel: Sendable { + public let name: String + public let repo: String + public let revision: String? + public let decider: any Decider + + public init(name: String, repo: String, revision: String?, decider: any Decider) { + self.name = name + self.repo = repo + self.revision = revision + self.decider = decider + } +} + +/// The routes: hosted Jev's two decision paths, the model list, and a health check +/// (see README.md). Decisions run one at a time (one GPU) in arrival order. +public final class Service: @unchecked Sendable { + public let models: [ServedModel] + public let defaultModel: String + let token: String? + let maxStateCharacters: Int? + let logBodies: Bool + let log: @Sendable (String) -> Void + private let queue: FIFOQueue + + public init(models: [ServedModel], defaultModel: String, token: String?, maxStateCharacters: Int?, + queueLimit: Int = 32, logBodies: Bool = false, log: @escaping @Sendable (String) -> Void = jevServeLog) { + self.models = models + self.defaultModel = defaultModel + self.token = token + self.maxStateCharacters = maxStateCharacters + self.logBodies = logBodies + self.log = log + self.queue = FIFOQueue(limit: queueLimit) + } + + public static let decisionPaths = ["/api/alpha/decisions", "/v1/classifier"] + + public func handle(_ req: HTTPRequest) async -> HTTPResponse { + let start = ContinuousClock.now + var note = "" + let response: HTTPResponse + let path = req.path.count > 1 && req.path.hasSuffix("/") ? String(req.path.dropLast()) : req.path + switch (req.method, path) { + case ("GET", "/health"): + response = .json(200, Data(#"{"status":"ok","models":[\#(models.map { jsonString($0.name) }.joined(separator: ","))]}"#.utf8)) + case (_, "/health"): + response = Self.error(405, "Use GET.") + case (let m, let p) where Self.decisionPaths.contains(p) || p == "/v1/models": + if let denied = authorize(req) { + response = denied + } else if p == "/v1/models" { + response = m == "GET" ? modelList() : Self.error(405, "Use GET.") + } else if m != "POST" { + response = Self.error(405, "Use POST with a JSON body.") + } else { + (response, note) = await decide(req) + } + default: + response = Self.error(404, "No route for \(req.method) \(req.path). jev-serve answers POST /api/alpha/decisions, POST /v1/classifier, GET /v1/models and GET /health.", type: "not_found") + } + let ms = Int((ContinuousClock.now - start) / .milliseconds(1)) + log("\(req.method) \(req.path)\(note.isEmpty ? "" : " " + note) → \(response.status) \(ms) ms") + if logBodies { + log(" request: \(String(decoding: req.body, as: UTF8.self))") + log(" response: \(String(decoding: response.body, as: UTF8.self))") + } + return response + } + + // MARK: Decisions + + private func decide(_ req: HTTPRequest) async -> (HTTPResponse, String) { + let wire: JevWire.Request + do { wire = try JevWire.decodeRequest(req.body) } catch { + return (Self.error(400, Self.message(error)), "") + } + let name = wire.model ?? defaultModel + guard let served = model(named: name) else { + return (Self.error(404, "jev-serve doesn't serve “\(name)”. Served: \(models.map(\.name).joined(separator: ", ")).", type: "not_found"), "") + } + let note = "\(served.name) \(wire.request.questions.count)q" + let l = served.decider.limits + let limits = DecisionLimits(maxQuestions: l.maxQuestions, maxChoiceCriteria: l.maxChoiceCriteria, + maxScoreLevels: l.maxScoreLevels, maxStateCharacters: maxStateCharacters ?? l.maxStateCharacters) + do { try wire.request.validate(against: limits) } catch { + return (Self.error(400, Self.message(error)), note) + } + guard await queue.acquire() else { + return (Self.error(503, "Busy: too many requests waiting. Try again shortly.", type: "overloaded"), note) + } + do { + var (decision, tokens) = try await served.decider.decide(wire.request) + await queue.release() + decision.usage = DecisionUsage(inputTokens: tokens, outputTokens: 0, cost: 0) + let body = try JevWire.encodeResponse(decision, model: served.repo, request: wire.request) + return (.json(200, body), note) + } catch let e as DecisionError { + await queue.release() + switch e { + case .invalidRequest: return (Self.error(400, Self.message(e)), note) + case .unavailable: return (Self.error(503, Self.message(e), type: "unavailable"), note) + default: return (Self.error(500, Self.message(e), type: "server_error"), note) + } + } catch { + await queue.release() + return (Self.error(500, Self.message(error), type: "server_error"), note) + } + } + + /// A served model by its config name, or by its repo id (with or without `openjev:`). + public func model(named name: String) -> ServedModel? { + let bare = name.hasPrefix("openjev:") ? String(name.dropFirst("openjev:".count)) : name + return models.first { $0.name == name } ?? models.first { $0.repo == bare } + } + + private func modelList() -> HTTPResponse { + let items = models.map { m in + #"{"id":\#(jsonString(m.name)),"object":"model","owned_by":"jev-serve","model":\#(jsonString(m.repo)),"# + + #""revision":\#(m.revision.map(jsonString) ?? "null"),"calibrated":\#(m.decider.calibrated)}"# + } + return .json(200, Data(#"{"object":"list","data":[\#(items.joined(separator: ","))]}"#.utf8)) + } + + // MARK: Auth + + private func authorize(_ req: HTTPRequest) -> HTTPResponse? { + guard let token else { return nil } + let given = req.headers["authorization"].map { $0.hasPrefix("Bearer ") ? String($0.dropFirst(7)) : $0 } ?? "" + return Self.constantTimeEqual(given, token) ? nil + : Self.error(401, "Send the token from jev-serve's config: Authorization: Bearer .", type: "unauthorized") + } + + static func constantTimeEqual(_ a: String, _ b: String) -> Bool { + let x = Array(a.utf8), y = Array(b.utf8) + var diff = UInt8(x.count == y.count ? 0 : 1) + for i in 0.. HTTPResponse { + .json(status, JevWire.encodeError(message: message, type: type, code: status)) + } + + static func message(_ error: Error) -> String { + if let e = error as? DecisionError { + switch e { + case .invalidRequest(let m), .malformedAnswer(let m), .backend(let m), .pairing(let m): return m + case .noRoute(let r): return "No route “\(r)”." + case .unavailable(let id, let a): return "\(id.rest) isn't available: \(a)" + @unknown default: return "\(e)" + } + } + return error.localizedDescription + } +} + +/// Runs one decision at a time, the rest waiting in arrival order, up to `limit` waiting. +actor FIFOQueue { + private let limit: Int + private var busy = false + private var waiting: [CheckedContinuation] = [] + + init(limit: Int) { self.limit = limit } + + func acquire() async -> Bool { + if !busy { busy = true; return true } + guard waiting.count < limit else { return false } + await withCheckedContinuation { waiting.append($0) } + return true // handed over by release(); still busy + } + + func release() { + if waiting.isEmpty { busy = false } else { waiting.removeFirst().resume() } + } +} + +/// A JSON string literal (slashes unescaped). +func jsonString(_ s: String) -> String { + let e = JSONEncoder() + e.outputFormatting = .withoutEscapingSlashes + return String(decoding: (try? e.encode(s)) ?? Data("\"\"".utf8), as: UTF8.self) +} + +private let logFormatter: DateFormatter = { + let f = DateFormatter() + f.dateFormat = "HH:mm:ss" + return f +}() + +/// One line on stderr, with the time. +public func jevServeLog(_ line: String) { + FileHandle.standardError.write(Data("\(logFormatter.string(from: Date())) \(line)\n".utf8)) +} diff --git a/examples/jev-serve/Tests/JevServeKitTests/JevServeKitTests.swift b/examples/jev-serve/Tests/JevServeKitTests/JevServeKitTests.swift new file mode 100644 index 0000000..1eb0298 --- /dev/null +++ b/examples/jev-serve/Tests/JevServeKitTests/JevServeKitTests.swift @@ -0,0 +1,271 @@ +import Foundation +import Network +import Testing +import LocalLMLabSDKCore +import LocalLMLabSDKRemote +@testable import JevServeKit + +// jev-serve without a GPU: a scripted decider behind the real routes and HTTP server, and the SDK's +// own hosted-Jev client (JevDecisionProvider) as the caller. Live MLX: `jev-serve --self-check`. + +/// Answers every question the same way: yes/no 0.8; a choice's first option at 0.9; a score's top +/// level. Optionally slow, failing, or with its own limits. +struct ScriptedDecider: Decider { + var limits = DecisionLimits(maxQuestions: 64, maxChoiceCriteria: 26, maxScoreLevels: 10) + var calibrated = true + var delay: Duration = .zero + var error: DecisionError? + var order: OrderLog? + + func decide(_ request: DecisionRequest) async throws -> (decision: Decision, inputTokens: Int?) { + if let order { await order.append(request.state) } + if delay > .zero { try await Task.sleep(for: delay) } + if let error { throw error } + var answers: [String: DecisionAnswer] = [:] + for q in request.questions { + switch q.kind { + case .noul: answers[q.id] = .noul(0.8) + case .choice(let c): + let rest = c.count > 1 ? 0.1 / Double(c.count - 1) : 0 + let probs = Dictionary(uniqueKeysWithValues: c.enumerated().map { ($0.element.key, $0.offset == 0 ? 0.9 : rest) }) + answers[q.id] = .choice(ChoiceAnswer(choice: c[0].key, confidence: 0.9, probabilities: probs)) + case .score(let levels): + var p = Array(repeating: 0.0, count: levels.count); p[p.count - 1] = 1 + answers[q.id] = .score(ScoreAnswer(score: Double(levels.count - 1), confidence: 1, probabilities: p, legend: levels)) + @unknown default: break + } + } + return (Decision(answers: answers, modelID: ModelID("openjev:test/model")!, fidelity: .tokenScored(calibrated: calibrated), + latency: .milliseconds(1)), 42) + } +} + +actor OrderLog { + private(set) var states: [DecisionState] = [] + func append(_ s: DecisionState) { states.append(s) } +} + +private func service(_ decider: ScriptedDecider = ScriptedDecider(), token: String? = nil, maxState: Int? = 20_000, + queueLimit: Int = 32) -> Service { + Service(models: [ServedModel(name: "support", repo: "mlx-community/Qwen3-4B-4bit", revision: "4dcb3d1abc", decider: decider)], + defaultModel: "support", token: token, maxStateCharacters: maxState, queueLimit: queueLimit, log: { _ in }) +} + +private let body = #""" +{"model":"support","state":"Nothing loads, my team is stuck.","questions":{ + "team":{"type":"choice","instructions":"Which team?","criteria":{"technical":"bugs","billing":"payments","account":"login"}}, + "refund":{"type":"noul","instructions":"Refund asked?"}, + "urgency":{"type":"score","instructions":"How urgent?","criteria":["can wait","soon","now"]}}} +"""# + +private func post(_ s: Service, _ path: String = "/v1/classifier", _ json: String = body, headers: [String: String] = [:]) async -> HTTPResponse { + await s.handle(HTTPRequest(method: "POST", path: path, headers: headers, body: Data(json.utf8))) +} + +private func errorMessage(_ r: HTTPResponse) -> String { JevWire.errorMessage(in: r.body) ?? "" } + +@Suite struct RouteTests { + @Test func bothDecisionPathsAnswerInHostedShape() async throws { + let s = service() + for path in Service.decisionPaths { + let r = await post(s, path) + #expect(r.status == 200) + let obj = try #require(try JSONSerialization.jsonObject(with: r.body) as? [String: Any]) + #expect(obj["model"] as? String == "mlx-community/Qwen3-4B-4bit") + #expect(obj["fidelity"] as? String == "tokenScored") + #expect(obj["calibrated"] as? Bool == true) + #expect((obj["usage"] as? [String: Any])?["input_tokens"] as? Int == 42) + let answers = try #require(obj["answers"] as? [String: Any]) + #expect((answers["team"] as? [String: Any])?["choice"] as? String == "technical") + #expect((answers["refund"] as? [String: Any])?["noul"] as? Double == 0.8) + } + } + + @Test func modelByNameRepoPrefixedOrDefault() async { + let s = service() + for model in ["support", "mlx-community/Qwen3-4B-4bit", "openjev:mlx-community/Qwen3-4B-4bit"] { + #expect(await post(s, "/v1/classifier", body.replacingOccurrences(of: #""model":"support""#, with: #""model":"\#(model)""#)).status == 200) + } + #expect(await post(s, "/v1/classifier", body.replacingOccurrences(of: #""model":"support","#, with: "")).status == 200) + let r = await post(s, "/v1/classifier", body.replacingOccurrences(of: #""model":"support""#, with: #""model":"gpt-9""#)) + #expect(r.status == 404) + #expect(errorMessage(r).contains("Served: support")) + } + + @Test func trailingSlashAndOtherRoutes() async throws { + let s = service() + #expect(await post(s, "/v1/classifier/").status == 200) + #expect(await s.handle(HTTPRequest(method: "GET", path: "/v1/classifier")).status == 405) + #expect(await s.handle(HTTPRequest(method: "GET", path: "/v1/chat/completions")).status == 404) + #expect(await s.handle(HTTPRequest(method: "POST", path: "/health")).status == 405) + let health = await s.handle(HTTPRequest(method: "GET", path: "/health")) + #expect(String(decoding: health.body, as: UTF8.self) == #"{"status":"ok","models":["support"]}"#) + let models = await s.handle(HTTPRequest(method: "GET", path: "/v1/models")) + let text = String(decoding: models.body, as: UTF8.self) + #expect(text.contains(#""model":"mlx-community/Qwen3-4B-4bit""#)) // slashes unescaped + #expect(text.contains(#""revision":"4dcb3d1abc""#) && text.contains(#""calibrated":true"#)) + } + + @Test func badRequestsAre400WithAReason() async { + let s = service(maxState: 50) + let cases: [(String, String)] = [ + ("not json", "isn't valid JSON"), + (#"{"state":"x"}"#, "questions"), + (#"{"state":"x","questions":{"a":{"type":"rank","instructions":"?"}}}"#, "rank"), + (#"{"state":"\#(String(repeating: "a", count: 60))","questions":{"a":{"type":"noul","instructions":"?"}}}"#, "at most 50"), + (#"{"state":"x","questions":{"a":{"type":"noul","instructions":"?"},"a":{"type":"noul","instructions":"?"}}}"#, "a"), + ] + for (json, fragment) in cases { + let r = await post(s, "/v1/classifier", json) + #expect(r.status == 400, "\(json)") + #expect(errorMessage(r).contains(fragment), "\(errorMessage(r))") + } + let tooMany = #"{"state":"x","questions":{"c":{"type":"choice","instructions":"?","criteria":[\#((0..<27).map { "\"k\($0)\"" }.joined(separator: ","))]}}}"# + #expect(await post(s, "/v1/classifier", tooMany).status == 400) + } + + @Test func deciderErrorsMapToStatuses() async { + let unavailable = DecisionError.unavailable(ModelID("openjev:test/model")!, .notDownloaded) + #expect(await post(service(ScriptedDecider(error: unavailable))).status == 503) + #expect(await post(service(ScriptedDecider(error: .invalidRequest("no")))).status == 400) + #expect(await post(service(ScriptedDecider(error: .backend("boom")))).status == 500) + } + + @Test func uncalibratedSaysSo() async throws { + let r = await post(service(ScriptedDecider(calibrated: false))) + let obj = try #require(try JSONSerialization.jsonObject(with: r.body) as? [String: Any]) + #expect(obj["calibrated"] as? Bool == false) + } +} + +@Suite struct AuthTests { + @Test func tokenGuardsEverythingButHealth() async { + let s = service(token: "secret-token") + #expect(await post(s).status == 401) + #expect(await post(s, headers: ["authorization": "Bearer wrong"]).status == 401) + #expect(await post(s, headers: ["authorization": "Bearer secret-token"]).status == 200) + #expect(await s.handle(HTTPRequest(method: "GET", path: "/v1/models")).status == 401) + #expect(await s.handle(HTTPRequest(method: "GET", path: "/health")).status == 200) + } + + @Test func constantTimeCompare() { + #expect(Service.constantTimeEqual("abc", "abc")) + #expect(!Service.constantTimeEqual("abc", "abd")) + #expect(!Service.constantTimeEqual("abc", "abcd")) + #expect(!Service.constantTimeEqual("", "a")) + } +} + +@Suite struct QueueTests { + @Test func decisionsRunOneAtATimeInArrivalOrder() async throws { + let log = OrderLog() + let s = service(ScriptedDecider(delay: .milliseconds(30), order: log)) + await withTaskGroup(of: Void.self) { group in + for i in 0..<5 { + group.addTask { + _ = await post(s, "/v1/classifier", body.replacingOccurrences(of: "Nothing loads, my team is stuck.", with: "input \(i)")) + } + try? await Task.sleep(for: .milliseconds(5)) // arrive in order + } + } + #expect(await log.states == (0..<5).map { DecisionState.text("input \($0)") }) + } + + @Test func fullQueueIs503() async { + let s = service(ScriptedDecider(delay: .milliseconds(200)), queueLimit: 1) + async let first = post(s) + try? await Task.sleep(for: .milliseconds(20)) + async let second = post(s) // waits + try? await Task.sleep(for: .milliseconds(20)) + let third = await post(s) // no room + #expect(third.status == 503) + #expect(await first.status == 200) + #expect(await second.status == 200) + } +} + +// MARK: - Over HTTP + +private func startServer(_ s: Service, maxBody: Int = 1_000_000) async throws -> (HTTPServer, URL) { + let server = try HTTPServer(host: "127.0.0.1", port: 0, maxBodyBytes: maxBody) { await s.handle($0) } + try await server.start() + return (server, URL(string: "http://127.0.0.1:\(try #require(server.port))")!) +} + +/// Sends raw bytes and returns the raw response (for malformed requests URLSession won't send). +private func raw(_ port: UInt16, _ bytes: Data) async throws -> String { + let conn = NWConnection(host: "127.0.0.1", port: NWEndpoint.Port(rawValue: port)!, using: .tcp) + conn.start(queue: .global()) + defer { conn.cancel() } + conn.send(content: bytes, completion: .idempotent) + return try await withCheckedThrowingContinuation { cont in + conn.receive(minimumIncompleteLength: 1, maximumLength: 64 * 1024) { data, _, _, error in + if let error { cont.resume(throwing: error) } else { cont.resume(returning: String(decoding: data ?? Data(), as: UTF8.self)) } + } + } +} + +@Suite(.serialized) struct HTTPTests { + @Test func keepAliveAndJSON() async throws { + let (server, base) = try await startServer(service()) + defer { server.stop() } + for _ in 0..<3 { + var req = URLRequest(url: base.appendingPathComponent("v1/classifier")) + req.httpMethod = "POST" + req.httpBody = Data(body.utf8) + let (data, response) = try await URLSession.shared.data(for: req) + #expect((response as? HTTPURLResponse)?.statusCode == 200) + #expect((response as? HTTPURLResponse)?.value(forHTTPHeaderField: "Content-Type") == "application/json") + #expect(JevWire.errorMessage(in: data) == nil) + } + } + + @Test func capsAndMalformedRequests() async throws { + let (server, base) = try await startServer(service(), maxBody: 1_000) + defer { server.stop() } + let port = try #require(server.port) + var big = URLRequest(url: base.appendingPathComponent("v1/classifier")) + big.httpMethod = "POST" + big.httpBody = Data(repeating: 0x61, count: 5_000) + let (_, r413) = try await URLSession.shared.data(for: big) + #expect((r413 as? HTTPURLResponse)?.statusCode == 413) + #expect(try await raw(port, Data("GARBAGE\r\n\r\n".utf8)).hasPrefix("HTTP/1.1 400")) + #expect(try await raw(port, Data("POST /v1/classifier HTTP/1.1\r\nTransfer-Encoding: chunked\r\n\r\n".utf8)).hasPrefix("HTTP/1.1 411")) + let hugeHeader = "GET /health HTTP/1.1\r\nX-Pad: " + String(repeating: "a", count: 20_000) + #expect(try await raw(port, Data(hugeHeader.utf8)).hasPrefix("HTTP/1.1 431")) + #expect(try await raw(port, Data("GET /health?x=1 HTTP/1.1\r\nHost: x\r\n\r\n".utf8)).contains(#""status":"ok""#)) + } + + /// The SDK's own hosted-Jev client, pointed at jev-serve: what it gets matches what the decider + /// answered, through HTTP and JevWire both ways, and the token and fidelity extension work. + @Test @MainActor func sdkClientRoundTrip() async throws { + let decider = ScriptedDecider() + let (server, base) = try await startServer(service(decider, token: "t0ken")) + defer { server.stop() } + let config = JevProviderConfig(scheme: "jevserve", displayName: "jev-serve", + endpoint: base.appendingPathComponent("v1/classifier"), auth: .apiKey("t0ken"), + fidelity: .tokenScored(calibrated: false), responseLimits: .default, timeout: 10) + let lab = LocalLMLab() + try lab.models.register(decision: JevDecisionProvider(try config.validated())) + lab.models.route(decision: "served", to: ModelID("jevserve:support")!) + let questions: [DecisionQuestion] = [ + .choice("team", "Which team?", criteria: ["technical": "bugs", "billing": "payments", "account": "login"]), + .noul("refund", "Refund asked?"), + .score("urgency", "How urgent?", criteria: ["can wait", "soon", "now"]), + ] + let request = DecisionRequest(state: .json(#"{"subject":"Refund","amount":12}"#), questions: questions) + let viaServer = try await lab.decide(route: "served", state: request.state, questions: questions) + let direct = try await decider.decide(request).decision + #expect(viaServer.answers == direct.answers) + #expect(viaServer.fidelity == .tokenScored(calibrated: true)) // from the server, over the config's default + #expect(viaServer.usage?.inputTokens == 42) + + // The wrong token is the provider's credential error, not a crash. + var wrong = config + wrong.auth = .apiKey("nope") + lab.models.replace(decision: JevDecisionProvider(wrong)) + await #expect(throws: DecisionError.self) { + _ = try await lab.decide(route: "served", state: .text("x"), questions: questions) + } + } +} diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index 2b109d5..02b2739 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -349,6 +349,16 @@ let result = try await lab.evaluate(route: "support", questions: set.questions, #expect(result.score("refund")!.accuracy >= 0.95) ``` +**Not a Swift app, or several apps?** Serve the tested setup over HTTP instead: **File → Export +Server Config…** writes a config for [jev-serve](../jev-serve/), which answers hosted Jev's API +(OpenRouter's and Featherless's) on this Mac. Code that already calls hosted Jev, in any language, +switches by changing its base URL: + +```bash +jev-serve --config jev-serve.json +curl -s http://127.0.0.1:8746/v1/classifier -H "Authorization: Bearer $TOKEN" -d @request.json +``` + Patterns that work: - **A threshold:** act on the answer when it's sure; otherwise do something safer (ask a diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index 86956a3..6ddb092 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -150,6 +150,9 @@ for your app: (CSV)…** loads one, e.g. from a spreadsheet. In your app's tests, `lab.evaluate(route:questions:answerSet:)` scores a backend against it. +- **File → Export Server Config…** writes a config for [jev-serve](../jev-serve/), which serves + the same tested setup over hosted Jev's HTTP API on this Mac, for apps in any language. + See [GUIDE.md](GUIDE.md) step 8. ## Reading the results diff --git a/skills/locallmlab-swift-app/references/example-selector.md b/skills/locallmlab-swift-app/references/example-selector.md index 1b50346..ba716b3 100644 --- a/skills/locallmlab-swift-app/references/example-selector.md +++ b/skills/locallmlab-swift-app/references/example-selector.md @@ -15,6 +15,7 @@ Pick the nearest reference app before inventing a new integration shape. - `examples/components-demo`: prebuilt Components UI for adding and managing MCP servers. - `examples/model-switch`: Core plus Remote plus Components for online providers, API keys, model selection, web search, and citations. - `examples/aiql`: SwiftUI "ask your data" app using MCP, `FileBackedTool`, data verbs, CSV output, sandboxing, and OAuth. +- `examples/jev-serve` (2.0): a command-line HTTP server answering hosted Jev's API (OpenRouter, Featherless) with OpenJev on this Mac, from JevDK's exported config. The reference for serving decisions to other processes or languages, and for `JevWire` (the wire format) on the server side. - `examples/jevdk` (2.0): JevDK, a SwiftUI playground for decision models: `lab.decide` across local `OpenJevDecisionProvider` (Inference) and hosted `JevDecisionProvider` (Remote), batch scoring against marked answers, `DecisionCalibration`, and `DecisionQuestionSet` / `DecisionAnswerSet` export. The reference for any decision-model integration; its `GUIDE.md` is the workflow. ## Toolkit Examples From 1dfc823cd3c22e523964a6ebac9bf96e1a77148b Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 06:29:36 -0700 Subject: [PATCH 53/73] Cascade republished 2.0.0-dev SDK checksums (Core, Inference, Remote, Claude); API surface MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All four 2.0.0-dev xcframeworks were rebuilt and re-uploaded on 2026-10-05 13:22–13:25 UTC (Core now has JevWire; Remote's JevDecisionProvider uses it). Values are the release assets' SHA-256 digests. Components and all 16 examples build against them, including jev-serve (13 tests pass; --self-check passes live on Qwen3-4B-4bit). api-surface.md regenerated from those binaries (interfaces byte-identical to the release assets): JevWire, and the Workspace / FileBackedTool changes from a733ab8. Co-Authored-By: Claude Opus 5.5 --- Components/Package.swift | 2 +- docs/api-surface.md | 80 +++++++++++++++----- examples/README.md | 6 +- examples/aiql/Package.swift | 4 +- examples/code-buddy/Package.swift | 4 +- examples/jevdk/Package.swift | 6 +- examples/mcp-chat/Package.swift | 2 +- examples/mlx-control-room/Package.swift | 4 +- examples/model-switch/Package.swift | 2 +- examples/os-matrix/Package.swift | 4 +- examples/plate-today-tools/Package.swift | 2 +- examples/plate-today/Package.swift | 2 +- examples/repo-qa-local/Package.swift | 4 +- examples/repo-qa/Package.swift | 2 +- examples/security-demo/Package.swift | 2 +- examples/vistanova/Package.swift | 4 +- examples/workspace-buddy-local/Package.swift | 4 +- examples/workspace-buddy/Package.swift | 2 +- 18 files changed, 88 insertions(+), 48 deletions(-) diff --git a/Components/Package.swift b/Components/Package.swift index 833a913..2850fff 100644 --- a/Components/Package.swift +++ b/Components/Package.swift @@ -33,7 +33,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" + checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" ), ] diff --git a/docs/api-surface.md b/docs/api-surface.md index 0a8c430..4485c99 100644 --- a/docs/api-surface.md +++ b/docs/api-surface.md @@ -2,7 +2,7 @@ > Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s. > Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check. -> SDK version: `"2.0.0-dev"` (the published `v2.0.0-dev` xcframeworks, republished 2026-10-04) · generated 2026-10-04T14:02Z +> SDK version: `"2.0.0-dev"` (the published `v2.0.0-dev` xcframeworks, republished 2026-10-05) · generated 2026-10-05T13:29Z ## LocalLMLabSDKCore @@ -866,13 +866,22 @@ public var blocked: String? public var isUsable: Bool public init(id: ModelID, capabilities: ModelCapabilities = [], notes: [String] = [], blocked: String? = nil) public static func == (a: ModelCapabilityReport, b: ModelCapabilityReport) -> Bool +public struct SaveAsPolicy : Sendable, Equatable +public var directory: String? +public var allowedExtensions: Set? +public var allowsHiddenPaths: Bool +public init(directory: String? = "raw", allowedExtensions: Set? = SaveAsPolicy.dataExtensions, allowsHiddenPaths: Bool = false) +public static let dataExtensions: Set +public static let `default`: SaveAsPolicy +public static let unrestricted: SaveAsPolicy +public static func == (a: SaveAsPolicy, b: SaveAsPolicy) -> Bool public struct FileBackedTool : Tool public typealias Arguments = GeneratedContent public let name: String public let description: String public let parameters: GenerationSchema public let toolOrigin: ToolOrigin -public init(name: String, description: String, argumentsJSONSchema: Data, root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, origin: ToolOrigin = .host, followUp: String = "read it back a window at a time with readFileRange", invoke: @escaping @Sendable (GeneratedContent) async -> String) throws +public init(name: String, description: String, argumentsJSONSchema: Data, root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, origin: ToolOrigin = .host, followUp: String = "read it back a window at a time with readFileRange", saveAsPolicy: SaveAsPolicy = .default, invoke: @escaping @Sendable (GeneratedContent) async -> String) throws nonisolated public func call(arguments: GeneratedContent) async throws -> String public typealias Output = String public enum GenerationErrorDescription @@ -913,6 +922,26 @@ public static func == (a: HostTranscript.ReplyInProgress, b: HostTranscript.Repl public enum JSONPath public static func value(at path: String, in root: Any) -> Any? public static func stringify(_ value: Any?) -> String +public enum JevWire +public struct Request : Sendable, Hashable +public var model: String? +public var request: DecisionRequest +public init(model: String?, request: DecisionRequest) +public static func == (a: JevWire.Request, b: JevWire.Request) -> Bool +public func hash(into hasher: inout Hasher) +public var hashValue: Int +public struct Response : Sendable +public var model: String? +public var answers: [String : DecisionAnswer] +public var usage: DecisionUsage? +public var fidelity: DecisionFidelity? +public var errorMessage: String? +public static func encodeRequest(_ request: DecisionRequest, model: String) throws -> Data +public static func decodeRequest(_ data: Data) throws -> JevWire.Request +public static func encodeResponse(_ decision: Decision, model: String, request: DecisionRequest) throws -> Data +public static func decodeResponse(_ data: Data, request: DecisionRequest) throws -> JevWire.Response +public static func encodeError(message: String, type: String, code: Int) -> Data +public static func errorMessage(in data: Data) -> String? @MainActor final public func decide(route: RouteName, state: DecisionState, questions: [DecisionQuestion]) async throws -> Decision public struct DecisionEvaluation : Sendable public struct QuestionScore : Sendable, Hashable @@ -1450,9 +1479,10 @@ public var root: URL public var previewCharacters: Int public var inlineCharacterLimit: Int? public var followUp: String -public init(root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, followUp: String = "read it back a window at a time with readFileRange") +public var saveAsPolicy: SaveAsPolicy +public init(root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, followUp: String = "read it back a window at a time with readFileRange", saveAsPolicy: SaveAsPolicy = .default) public static func == (a: MCPFileBackedOutput, b: MCPFileBackedOutput) -> Bool -public static func mcp(descriptor: MCPToolDescriptor, manager: MCPServerManager, root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, followUp: String = "read it back a window at a time with readFileRange") throws -> FileBackedTool +public static func mcp(descriptor: MCPToolDescriptor, manager: MCPServerManager, root: URL, previewCharacters: Int = 600, inlineCharacterLimit: Int? = nil, followUp: String = "read it back a window at a time with readFileRange", saveAsPolicy: SaveAsPolicy = .default) throws -> FileBackedTool public struct MCPResourceLink : Sendable, Codable, Equatable public var uri: String public var name: String? @@ -2043,7 +2073,16 @@ public var hunksApplied: Int public static func == (a: WorkspaceAccess.PatchResult, b: WorkspaceAccess.PatchResult) -> Bool public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws -public static func applyPatch(in root: URL, diff: String) -> Result +public struct WorkspaceLimits : Sendable, Equatable +public var maxReadBytes: Int +public var maxReturnedCharacters: Int +public var maxFileBytes: Int +public var maxRegexInputBytes: Int +public var maxScanBytes: Int +public var maxWriteBytes: Int +public init(maxReadBytes: Int = 8 << 20, maxReturnedCharacters: Int = 2_000_000, maxFileBytes: Int = 64 << 20, maxRegexInputBytes: Int = 8 << 10, maxScanBytes: Int = 256 << 20, maxWriteBytes: Int = 64 << 20) +public static let `default`: WorkspaceAccess.WorkspaceLimits +public static func == (a: WorkspaceAccess.WorkspaceLimits, b: WorkspaceAccess.WorkspaceLimits) -> Bool public struct SearchMatch : Codable, Sendable, Equatable public var path: String public var line: Int @@ -2051,9 +2090,14 @@ public var text: String public static func == (a: WorkspaceAccess.SearchMatch, b: WorkspaceAccess.SearchMatch) -> Bool public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws -public static func search(in root: URL, query: String, isRegex: Bool = false, include: String? = nil, maxResults: Int = 200) -> Result<[WorkspaceAccess.SearchMatch], WorkspaceAccess.WorkspaceError> -public static func tree(in root: URL, subpath: String? = nil, maxDepth: Int = 2) -> Result -public static func readFileRange(in root: URL, path: String, offset: Int, limit: Int) -> Result +public struct SearchResult : Codable, Sendable, Equatable +public var matches: [WorkspaceAccess.SearchMatch] +public var skippedTooLarge: [String] +public var scanBudgetExhausted: Bool +public var hitResultCap: Bool +public static func == (a: WorkspaceAccess.SearchResult, b: WorkspaceAccess.SearchResult) -> Bool +public func encode(to encoder: any Encoder) throws +public init(from decoder: any Decoder) throws public enum WorkspaceAccess public struct WorkspaceError : Error public var message: String @@ -2064,17 +2108,13 @@ public var modifiedDate: Date? public var size: Int? public func encode(to encoder: any Encoder) throws public init(from decoder: any Decoder) throws -public static let maxReadCharacters: Int -public static func listFiles(in root: URL, subpath: String?) -> Result<[WorkspaceAccess.WorkspaceEntry], WorkspaceAccess.WorkspaceError> -public static func readFile(in root: URL, path: String) -> Result -public static func writeFile(in root: URL, path: String, contents: String, overwrite: Bool = false, append: Bool = false) -> Result -public static func editFile(in root: URL, path: String, oldString: String, newString: String, replaceAll: Bool) -> Result -public static func deleteFile(in root: URL, path: String) -> Result +public static var maxReadCharacters: Int +public var errorDescription: String? public struct SearchWorkspaceTool : Tool public static let defaultDescription: String public let name: String public let description: String -public init(root: URL, description: String? = nil) +public init(root: URL, description: String? = nil, limits: WorkspaceAccess.WorkspaceLimits = .default) public struct Arguments public var query: String public var isRegex: Bool? @@ -2114,7 +2154,7 @@ public struct ApplyPatchTool : Tool public static let defaultDescription: String public let name: String public let description: String -public init(root: URL, description: String? = nil) +public init(root: URL, description: String? = nil, limits: WorkspaceAccess.WorkspaceLimits = .default) public struct Arguments public var diff: String public init(diff: String) @@ -2131,7 +2171,7 @@ public struct ReadFileRangeTool : Tool public static let defaultDescription: String public let name: String public let description: String -public init(root: URL, description: String? = nil) +public init(root: URL, description: String? = nil, limits: WorkspaceAccess.WorkspaceLimits = .default) public struct Arguments public var path: String public var offset: Int @@ -2173,7 +2213,7 @@ public struct ReadWorkspaceFileTool : Tool public static let defaultDescription: String public let name: String public let description: String -public init(root: URL, description: String? = nil) +public init(root: URL, description: String? = nil, limits: WorkspaceAccess.WorkspaceLimits = .default) public struct Arguments public var path: String public init(path: String) @@ -2190,7 +2230,7 @@ public struct WriteWorkspaceFileTool : Tool public static let defaultDescription: String public let name: String public let description: String -public init(root: URL, description: String? = nil) +public init(root: URL, description: String? = nil, limits: WorkspaceAccess.WorkspaceLimits = .default) public struct Arguments public var path: String public var contents: String @@ -2213,7 +2253,7 @@ public struct EditWorkspaceFileTool : Tool public static let defaultDescription: String public let name: String public let description: String -public init(root: URL, description: String? = nil) +public init(root: URL, description: String? = nil, limits: WorkspaceAccess.WorkspaceLimits = .default) public struct Arguments public var path: String public var oldString: String diff --git a/examples/README.md b/examples/README.md index 5831813..984e76b 100644 --- a/examples/README.md +++ b/examples/README.md @@ -177,7 +177,7 @@ let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease(url: "…/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2…"), "2.0.0-dev": SDKRelease(url: "…/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "532bc89a…"), + checksum: "c741066f…"), ] ``` @@ -189,9 +189,9 @@ distinct `.xcframework.zip.sha256`; don't reuse Core's for another module): ```bash curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip.sha256 -# → 532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e +# → c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKClaude-2.0.0-dev.xcframework.zip.sha256 -# → fc01751adada70e60e6fc8ad6e74415712fe9371eb2b138d0ff56905a12addd0 +# → 65ec39fe1f2965041e05793e86850f624d1f8bfd6ebbd8111b9c06f68c1975f2 ``` Then point `defaultSDKVersion` at it (works everywhere, Xcode included) or pass diff --git a/examples/aiql/Package.swift b/examples/aiql/Package.swift index c7c896a..5158ec7 100644 --- a/examples/aiql/Package.swift +++ b/examples/aiql/Package.swift @@ -42,9 +42,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/code-buddy/Package.swift b/examples/code-buddy/Package.swift index 534c83c..a85cfef 100644 --- a/examples/code-buddy/Package.swift +++ b/examples/code-buddy/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/jevdk/Package.swift b/examples/jevdk/Package.swift index 8ff7554..403a181 100644 --- a/examples/jevdk/Package.swift +++ b/examples/jevdk/Package.swift @@ -38,11 +38,11 @@ let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "c9d193b5d8b0864b562ccbeee94ff5eb8915d5428f3c4dcc38c06585e737790b", + remoteChecksum: "a48098d692084ccde5db7c9e2e39806b579b387147a36546995e606baa51099e", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift index be4d1bc..c0b61c2 100644 --- a/examples/mcp-chat/Package.swift +++ b/examples/mcp-chat/Package.swift @@ -36,7 +36,7 @@ let defaultSDKVersion = "2.0.0-dev" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-dev": SDKRelease( inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/mlx-control-room/Package.swift b/examples/mlx-control-room/Package.swift index fca63b9..e7a5949 100644 --- a/examples/mlx-control-room/Package.swift +++ b/examples/mlx-control-room/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/model-switch/Package.swift b/examples/model-switch/Package.swift index 8caea66..09e7f67 100644 --- a/examples/model-switch/Package.swift +++ b/examples/model-switch/Package.swift @@ -37,7 +37,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "c9d193b5d8b0864b562ccbeee94ff5eb8915d5428f3c4dcc38c06585e737790b" + remoteChecksum: "a48098d692084ccde5db7c9e2e39806b579b387147a36546995e606baa51099e" ), ] diff --git a/examples/os-matrix/Package.swift b/examples/os-matrix/Package.swift index 2d43baf..1aa39e2 100644 --- a/examples/os-matrix/Package.swift +++ b/examples/os-matrix/Package.swift @@ -33,9 +33,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/plate-today-tools/Package.swift b/examples/plate-today-tools/Package.swift index 875ffb4..3ff6fdf 100644 --- a/examples/plate-today-tools/Package.swift +++ b/examples/plate-today-tools/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" + checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" ), ] diff --git a/examples/plate-today/Package.swift b/examples/plate-today/Package.swift index 9e13565..1345885 100644 --- a/examples/plate-today/Package.swift +++ b/examples/plate-today/Package.swift @@ -29,7 +29,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" + checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" ), ] diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index 8161c99..b3b0fb5 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -37,9 +37,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/repo-qa/Package.swift b/examples/repo-qa/Package.swift index 6c64e0e..a571878 100644 --- a/examples/repo-qa/Package.swift +++ b/examples/repo-qa/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" + checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" ), ] diff --git a/examples/security-demo/Package.swift b/examples/security-demo/Package.swift index 78e3ac9..a890738 100644 --- a/examples/security-demo/Package.swift +++ b/examples/security-demo/Package.swift @@ -38,7 +38,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "c9d193b5d8b0864b562ccbeee94ff5eb8915d5428f3c4dcc38c06585e737790b" + remoteChecksum: "a48098d692084ccde5db7c9e2e39806b579b387147a36546995e606baa51099e" ), ] diff --git a/examples/vistanova/Package.swift b/examples/vistanova/Package.swift index 2f7d11d..d14eea1 100644 --- a/examples/vistanova/Package.swift +++ b/examples/vistanova/Package.swift @@ -35,9 +35,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/workspace-buddy-local/Package.swift b/examples/workspace-buddy-local/Package.swift index 96448e4..2ab9713 100644 --- a/examples/workspace-buddy-local/Package.swift +++ b/examples/workspace-buddy-local/Package.swift @@ -39,9 +39,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e", + coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "32f832bc103fd0e689e296f24539987973d3c5f989a65c210196b9f3c771d9b4" + inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" ), ] diff --git a/examples/workspace-buddy/Package.swift b/examples/workspace-buddy/Package.swift index 661b7c8..c69fc10 100644 --- a/examples/workspace-buddy/Package.swift +++ b/examples/workspace-buddy/Package.swift @@ -30,7 +30,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-dev": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "532bc89ab6d877e565d93f87ec9a4058a484910a195b5436372d5ce12b55e74e" + checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" ), ] From b8f7cbe479af05d53567fb900b07f6aca49b685c Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 06:46:10 -0700 Subject: [PATCH 54/73] jev-serve release zip, --version; JevDK release script finds NOTICE in either repo - jev-serve/scripts/release_macos.sh: builds against the published SDK binaries, stages the binary with LocalLMLabSDKCore/Inference frameworks, NOTICE and a README, signs inside-out with the hardened runtime, smoke-tests, zips (no AppleDouble entries) and notarizes; .sha256 alongside. - jev-serve --version (stamped by the release script; "dev" from source). - jevdk/scripts/release_macos.sh: the SDK NOTICE is one level up in ancientcomputing/jevdk, two in locallm; it looked only two up. - jev-serve logging is a @Sendable function value (no concurrency warning). - JevDK README: the SDK-version link reads right in both repos. scripts/sync-jevdk-repo.sh: copies examples/jevdk and examples/jev-serve into a clone of ancientcomputing/jevdk (unchanged), refreshes NOTICE, builds both and runs jev-serve's tests. Co-Authored-By: Claude Opus 5.5 --- examples/jev-serve/.gitignore | 2 + .../jev-serve/Sources/JevServe/Version.swift | 3 + .../jev-serve/Sources/JevServe/main.swift | 2 + .../Sources/JevServeKit/Service.swift | 4 +- examples/jev-serve/scripts/release_macos.sh | 139 ++++++++++++++++++ examples/jevdk/README.md | 2 +- examples/jevdk/scripts/release_macos.sh | 13 +- scripts/sync-jevdk-repo.sh | 41 ++++++ 8 files changed, 200 insertions(+), 6 deletions(-) create mode 100644 examples/jev-serve/Sources/JevServe/Version.swift create mode 100755 examples/jev-serve/scripts/release_macos.sh create mode 100755 scripts/sync-jevdk-repo.sh diff --git a/examples/jev-serve/.gitignore b/examples/jev-serve/.gitignore index 5104546..15ddf45 100644 --- a/examples/jev-serve/.gitignore +++ b/examples/jev-serve/.gitignore @@ -1,2 +1,4 @@ .build/ +build/ +dist/ jev-serve.json diff --git a/examples/jev-serve/Sources/JevServe/Version.swift b/examples/jev-serve/Sources/JevServe/Version.swift new file mode 100644 index 0000000..67f1ba6 --- /dev/null +++ b/examples/jev-serve/Sources/JevServe/Version.swift @@ -0,0 +1,3 @@ +// The jev-serve version, stamped by scripts/release_macos.sh for a release build and restored +// afterwards. "dev" for anything built from source. +let jevServeVersion = "dev" diff --git a/examples/jev-serve/Sources/JevServe/main.swift b/examples/jev-serve/Sources/JevServe/main.swift index 91dc56e..901628f 100644 --- a/examples/jev-serve/Sources/JevServe/main.swift +++ b/examples/jev-serve/Sources/JevServe/main.swift @@ -26,6 +26,7 @@ let usage = """ --port port (default from the config, else 8746) --no-download fail instead of downloading a missing model --log-bodies also log request and response bodies (they hold your inputs) + --version print the version and exit --self-check start on a spare port, ask every model the same questions over HTTP and directly, check the answers match, and exit (0 = all match) --help @@ -53,6 +54,7 @@ func flag(_ name: String) -> Bool { } setvbuf(stdout, nil, _IOLBF, 0) // line-buffered, so the banner shows when piped or logged if flag("--help") || flag("-h") { print(usage); exit(0) } +if flag("--version") { print("jev-serve \(jevServeVersion) (LocalLM Lab SDK \(LocalLMLabSDKVersion.current))"); exit(0) } let configPath = option("--config") let modelOption = option("--model") let hostOption = option("--host") diff --git a/examples/jev-serve/Sources/JevServeKit/Service.swift b/examples/jev-serve/Sources/JevServeKit/Service.swift index 4d88d16..eb372c3 100644 --- a/examples/jev-serve/Sources/JevServeKit/Service.swift +++ b/examples/jev-serve/Sources/JevServeKit/Service.swift @@ -217,13 +217,13 @@ func jsonString(_ s: String) -> String { return String(decoding: (try? e.encode(s)) ?? Data("\"\"".utf8), as: UTF8.self) } -private let logFormatter: DateFormatter = { +nonisolated(unsafe) private let logFormatter: DateFormatter = { // DateFormatter is thread-safe for formatting let f = DateFormatter() f.dateFormat = "HH:mm:ss" return f }() /// One line on stderr, with the time. -public func jevServeLog(_ line: String) { +public let jevServeLog: @Sendable (String) -> Void = { line in FileHandle.standardError.write(Data("\(logFormatter.string(from: Date())) \(line)\n".utf8)) } diff --git a/examples/jev-serve/scripts/release_macos.sh b/examples/jev-serve/scripts/release_macos.sh new file mode 100755 index 0000000..b274b57 --- /dev/null +++ b/examples/jev-serve/scripts/release_macos.sh @@ -0,0 +1,139 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Builds a Developer ID–signed, notarized jev-serve and zips it with the SDK frameworks it loads. +# +# VERSION=0.2.0 \ +# APP_IDENTITY="Developer ID Application: Your Name (TEAMID)" \ +# KEYCHAIN_PROFILE=notary-profile \ +# scripts/release_macos.sh +# +# VERSION required, x.y.z — printed by `jev-serve --version` +# APP_IDENTITY required, a "Developer ID Application" identity (name or SHA-1 hash; use the +# hash when several identities share a name). SIGN_IDENTITY also accepted. +# KEYCHAIN_PROFILE notarytool profile (xcrun notarytool store-credentials ); required +# unless NOTARIZE=0. NOTARY_PROFILE also accepted. +# TEAM_ID optional, passed to notarytool +# NOTARIZE 1 (default) | 0 +# +# The zip holds one folder: the jev-serve binary, the two frameworks next to it that it loads by +# @executable_path (LocalLMLabSDKCore, LocalLMLabSDKInference — MLX's compiled shaders are inside +# Inference), the SDK's NOTICE and a short README. Everything is signed inside-out with the +# hardened runtime. A zip can't be stapled: macOS checks the notarization online the first time +# the binary runs. +# +# Artifacts: dist/jev-serve--arm64.zip (+ .sha256). Scratch in build/release. + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +APP_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" + +VERSION="${VERSION:-}" +APP_IDENTITY="${APP_IDENTITY:-${SIGN_IDENTITY:-}}" +KEYCHAIN_PROFILE="${KEYCHAIN_PROFILE:-${NOTARY_PROFILE:-}}" +TEAM_ID="${TEAM_ID:-}" +NOTARIZE="${NOTARIZE:-1}" + +DIST_DIR="$APP_ROOT/dist" +BUILD_DIR="$APP_ROOT/build/release" +NAME="jev-serve-${VERSION}" +STAGE="$BUILD_DIR/$NAME" +ZIP_PATH="$DIST_DIR/${NAME}-arm64.zip" +VERSION_FILE="$APP_ROOT/Sources/JevServe/Version.swift" + +die() { echo "$*" >&2; exit 1; } +sign() { codesign --force --options runtime --timestamp --sign "$APP_IDENTITY" "$@"; } + +[[ "$VERSION" =~ ^[0-9]+\.[0-9]+\.[0-9]+([.-][0-9A-Za-z.-]+)?$ ]] || die "Set VERSION=x.y.z (got '${VERSION}')" +[[ -n "$APP_IDENTITY" ]] || die "APP_IDENTITY is required" +if [[ "$NOTARIZE" == "1" ]]; then [[ -n "$KEYCHAIN_PROFILE" ]] || die "KEYCHAIN_PROFILE is required (or NOTARIZE=0)"; fi +for c in swift codesign ditto shasum xcrun xattr; do command -v "$c" >/dev/null || die "$c is required"; done + +security find-identity -v -p codesigning | grep -F "$APP_IDENTITY" >/dev/null \ + || die "APP_IDENTITY is not installed or not valid for codesigning: $APP_IDENTITY" +# Ambiguous only if the name matches different certificates (the same one is often listed several +# times, from the login, System and iCloud keychains, with the same SHA-1). +[[ "$(security find-identity -v -p codesigning | grep -F "$APP_IDENTITY" | awk '{print $2}' | sort -u | wc -l | tr -d ' ')" -le 1 ]] \ + || die "APP_IDENTITY matches more than one certificate; pass its SHA-1 hash instead" +if [[ "$NOTARIZE" == "1" ]]; then + xcrun notarytool history --keychain-profile "$KEYCHAIN_PROFILE" >/dev/null 2>&1 \ + || die "KEYCHAIN_PROFILE is not usable: $KEYCHAIN_PROFILE (create it with: xcrun notarytool store-credentials $KEYCHAIN_PROFILE)" +fi + +echo "Cleaning release artifacts..." +rm -rf "$BUILD_DIR" "$ZIP_PATH" "$ZIP_PATH.sha256" +mkdir -p "$STAGE" "$DIST_DIR" + +# Stamp the version for this build only; put the source back however the script ends. +cp "$VERSION_FILE" "$BUILD_DIR/Version.swift.orig" +restore_version() { cp "$BUILD_DIR/Version.swift.orig" "$VERSION_FILE" 2>/dev/null || true; } +trap restore_version EXIT +sed -i '' "s/^let jevServeVersion = .*/let jevServeVersion = \"$VERSION\"/" "$VERSION_FILE" + +echo "Building jev-serve (release, arm64)..." +( cd "$APP_ROOT" && swift build -c release --arch arm64 --product jev-serve ) +BIN_DIR="$(cd "$APP_ROOT" && swift build -c release --arch arm64 --show-bin-path)" +restore_version + +echo "Staging $NAME..." +cp "$BIN_DIR/jev-serve" "$STAGE/jev-serve" +for fw in LocalLMLabSDKCore LocalLMLabSDKInference; do + [[ -d "$BIN_DIR/$fw.framework" ]] || die "missing $fw.framework next to the binary in $BIN_DIR. This script packages the published SDK binaries: run it from ancientcomputing/locallm or ancientcomputing/jevdk, not a source-built SDK." + ditto "$BIN_DIR/$fw.framework" "$STAGE/$fw.framework" +done +# Apache-2.0 §4(d): the SDK's NOTICE (it bundles the MLX stack). At the repository root: one level +# up in ancientcomputing/jevdk, two in ancientcomputing/locallm. +for candidate in "$APP_ROOT/../NOTICE" "$APP_ROOT/../../NOTICE"; do + if [[ -f "$candidate" ]]; then cp "$candidate" "$STAGE/NOTICE"; break; fi +done +[[ -f "$STAGE/NOTICE" ]] || echo "warning: no NOTICE found next to this repository; the zip ships without it" >&2 +cat > "$STAGE/README.txt" < --keychain-profile $KEYCHAIN_PROFILE)" +else + echo "Skipping notarization because NOTARIZE=$NOTARIZE" +fi + +# Bare filename in the checksum, so `shasum -a 256 -c .sha256` works where it's downloaded. +( cd "$DIST_DIR" && shasum -a 256 "$(basename "$ZIP_PATH")" > "$(basename "$ZIP_PATH").sha256" ) + +echo "Release artifacts:" +echo "$ZIP_PATH" +echo "$ZIP_PATH.sha256" diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index 6ddb092..c02024d 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -12,7 +12,7 @@ Needs an Apple-silicon Mac, macOS 27 and Xcode 27 (Swift 6.4). `Package.swift` b `2.0.0-dev` with no further setup (the decision API is new in 2.0): `LocalLMLabSDKCore`, `LocalLMLabSDKRemote` and `LocalLMLabSDKInference` (the MLX runtime) come from that GitHub Release. No Metal Toolchain needed; the prebuilt Inference xcframework bundles the compiled shaders. To -build against another release, set `LOCALLM_SDK_VERSION` (see [the examples README](../README.md)). +build against another release, set `LOCALLM_SDK_VERSION` (see [the README one folder up](../README.md)). **Or open it in Xcode and Run:** `open -a Xcode JevDK.xcodeproj`, scheme **JevDK**. It's ad-hoc signed ("Sign to Run Locally") and not sandboxed, like the `swift build` binary, so it shares your diff --git a/examples/jevdk/scripts/release_macos.sh b/examples/jevdk/scripts/release_macos.sh index e00921b..6026574 100755 --- a/examples/jevdk/scripts/release_macos.sh +++ b/examples/jevdk/scripts/release_macos.sh @@ -158,10 +158,17 @@ ditto "$ARCHIVE_PATH/Products/Applications/${APP_NAME}.app" "$APP_DIR" /usr/libexec/PlistBuddy -c "Set :CFBundleShortVersionString $VERSION" "$CONTENTS_DIR/Info.plist" /usr/libexec/PlistBuddy -c "Set :CFBundleVersion $VERSION" "$CONTENTS_DIR/Info.plist" -# Apache-2.0 §4(d): carry the SDK's NOTICE (it bundles the MLX stack) inside the app. -if [[ -f "$APP_ROOT/../../NOTICE" ]]; then +# Apache-2.0 §4(d): carry the SDK's NOTICE (it bundles the MLX stack) inside the app. It's at the +# repository root: one level up in ancientcomputing/jevdk, two in ancientcomputing/locallm. +NOTICE_FILE="" +for candidate in "$APP_ROOT/../NOTICE" "$APP_ROOT/../../NOTICE"; do + if [[ -f "$candidate" ]]; then NOTICE_FILE="$candidate"; break; fi +done +if [[ -n "$NOTICE_FILE" ]]; then mkdir -p "$CONTENTS_DIR/Resources" - cp "$APP_ROOT/../../NOTICE" "$CONTENTS_DIR/Resources/NOTICE" + cp "$NOTICE_FILE" "$CONTENTS_DIR/Resources/NOTICE" +else + echo "warning: no NOTICE found next to this repository; the app ships without it" >&2 fi # The published xcframework zips carry AppleDouble/xattr detritus that breaks signing. diff --git a/scripts/sync-jevdk-repo.sh b/scripts/sync-jevdk-repo.sh new file mode 100755 index 0000000..6c9b666 --- /dev/null +++ b/scripts/sync-jevdk-repo.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +set -euo pipefail + +# sync-jevdk-repo.sh — copy JevDK and jev-serve from this repo into a clone of +# ancientcomputing/jevdk, the standalone repo end users download them from. +# +# scripts/sync-jevdk-repo.sh ../jevdk +# +# One-way: this repo (examples/jevdk, examples/jev-serve) is the source of truth, and both folders +# are copied unchanged into /jevdk and /jev-serve, so they build there exactly as +# they do here (jev-serve takes the SDK binaries from ../jevdk). The clone's own files (README.md, +# NOTICE, SOURCE.md, scripts/) are left alone, except NOTICE, which is refreshed from this repo's. +# Then it builds both, runs jev-serve's tests, and shows what changed. It never commits or pushes. + +cd "$(git rev-parse --show-toplevel)" +SRC="$PWD" +DEST="${1:-}" +[[ -n "$DEST" && -d "$DEST/.git" ]] || { echo "usage: scripts/sync-jevdk-repo.sh " >&2; exit 1; } +DEST="$(cd "$DEST" && pwd)" +git -C "$DEST" remote get-url origin | grep -q "ancientcomputing/jevdk" \ + || { echo "$DEST isn't a clone of ancientcomputing/jevdk" >&2; exit 1; } + +echo "== Checking this repo for private references first" +./scripts/check-public-hygiene.sh + +EXCLUDES=(--exclude .build/ --exclude .swiftpm/ --exclude build/ --exclude dist/ --exclude Package.resolved + --exclude xcuserdata/ --exclude .DS_Store --exclude jev-serve.json --exclude .env --exclude 'key.env') +for d in jevdk jev-serve; do + echo "== Copying examples/$d → $DEST/$d" + rsync -a --delete "${EXCLUDES[@]}" "$SRC/examples/$d/" "$DEST/$d/" +done +cp "$SRC/NOTICE" "$DEST/NOTICE" + +echo "== Building" +( cd "$DEST/jevdk" && swift build -c release 2>&1 | tail -1 ) +( cd "$DEST/jev-serve" && swift build -c release 2>&1 | tail -1 ) +echo "== jev-serve tests" +( cd "$DEST/jev-serve" && swift test 2>&1 | grep -E "Test run with|✘|error:" | tail -3 ) + +echo "== Changes in $DEST (review, then commit and push yourself)" +git -C "$DEST" status --short From 0ef3d4c7c3b2bf771cdb083ae210be35bc59fddb Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 07:16:31 -0700 Subject: [PATCH 55/73] =?UTF-8?q?JevDK=20carries=20jev-serve;=20JevDK=20?= =?UTF-8?q?=E2=86=92=20Install=20jev-serve=20Command=E2=80=A6?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A jev-serve zip downloaded in a browser fails Gatekeeper: its loose frameworks are quarantined and can't carry a notarization ticket, so macOS refuses to load them. The DMG script now builds jev-serve from ../jev-serve and embeds it at JevDK.app/Contents/MacOS/jev-serve, using the app's frameworks (rpath @executable_path/../Frameworks), signed with the app and covered by its stapled ticket. A new app-menu item links it into /usr/local/bin (administrator prompt), with a copy-the-command fallback and a move-to-Applications check. Docs: jev-serve comes with JevDK. Co-Authored-By: Claude Opus 5.5 --- examples/jev-serve/README.md | 13 ++++- examples/jevdk/GUIDE.md | 3 +- examples/jevdk/README.md | 3 +- examples/jevdk/Sources/JevDK/JevDKApp.swift | 62 +++++++++++++++++++++ examples/jevdk/scripts/release_macos.sh | 25 ++++++++- 5 files changed, 102 insertions(+), 4 deletions(-) diff --git a/examples/jev-serve/README.md b/examples/jev-serve/README.md index af8856f..bb4f491 100644 --- a/examples/jev-serve/README.md +++ b/examples/jev-serve/README.md @@ -9,12 +9,23 @@ changing its base URL. New to decision models? Start with [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html) and the [JevDK guide](https://thisbrain.ai/locallm/jdk-guide.html). +**The easy way: it comes with JevDK.** Download JevDK from +[ancientcomputing/jevdk](https://github.com/ancientcomputing/jevdk/releases/latest), move it to +Applications, then choose **JevDK → Install jev-serve Command…**. That puts `jev-serve` on your +PATH (signed and notarized with the app; it updates when JevDK does): + +```bash +jev-serve --config jev-serve.json +``` + +**From source:** + ```bash swift build -c release .build/release/jev-serve --config jev-serve.json ``` -Needs an Apple-silicon Mac, macOS 27 and Xcode 27 (Swift 6.4). +Needs an Apple-silicon Mac with macOS 27; building needs Xcode 27 (Swift 6.4). ## 1. Make a config in JevDK diff --git a/examples/jevdk/GUIDE.md b/examples/jevdk/GUIDE.md index 02b2739..9a10304 100644 --- a/examples/jevdk/GUIDE.md +++ b/examples/jevdk/GUIDE.md @@ -352,7 +352,8 @@ let result = try await lab.evaluate(route: "support", questions: set.questions, **Not a Swift app, or several apps?** Serve the tested setup over HTTP instead: **File → Export Server Config…** writes a config for [jev-serve](../jev-serve/), which answers hosted Jev's API (OpenRouter's and Featherless's) on this Mac. Code that already calls hosted Jev, in any language, -switches by changing its base URL: +switches by changing its base URL. jev-serve comes with the JevDK download: choose **JevDK → +Install jev-serve Command…** once to put it on your PATH. ```bash jev-serve --config jev-serve.json diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index c02024d..07af656 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -151,7 +151,8 @@ for your app: In your app's tests, `lab.evaluate(route:questions:answerSet:)` scores a backend against it. - **File → Export Server Config…** writes a config for [jev-serve](../jev-serve/), which serves - the same tested setup over hosted Jev's HTTP API on this Mac, for apps in any language. + the same tested setup over hosted Jev's HTTP API on this Mac, for apps in any language. The + JevDK download includes jev-serve: **JevDK → Install jev-serve Command…** puts it on your PATH. See [GUIDE.md](GUIDE.md) step 8. diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index 4ec263c..f0be710 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -47,6 +47,7 @@ struct JevDKApp: App { // questions for your app (the SDK's DecisionQuestionSet), answers (CSV), and results (CSV). CommandGroup(replacing: .appInfo) { Button("About JevDK") { AboutPanel.show() } + Button("Install jev-serve Command…") { JevServeInstaller.install() } } CommandGroup(replacing: .newItem) { Section("Workspace") { @@ -370,6 +371,67 @@ struct BackendsSheet: View { } } +/// JevDK → Install jev-serve Command…: the released app carries jev-serve (Contents/MacOS), signed +/// and notarized with it. This links it into /usr/local/bin (on the default PATH), asking for an +/// administrator password; if that doesn't work, it shows the command to run instead. +enum JevServeInstaller { + static let target = "/usr/local/bin/jev-serve" + + static func install() { + guard let tool = Bundle.main.url(forAuxiliaryExecutable: "jev-serve")?.path, + FileManager.default.isExecutableFile(atPath: tool) else { + inform("jev-serve isn't in this copy of JevDK", + "The JevDK download includes jev-serve; a JevDK built from source doesn't. Build it from the jev-serve folder of the source: swift build -c release.") + return + } + let appPath = Bundle.main.bundlePath + if appPath.contains("/AppTranslocation/") || appPath.hasPrefix("/Volumes/") { + inform("Move JevDK to Applications first", + "JevDK is running from the disk image or a temporary location, so a command installed now would stop working. Drag JevDK to Applications, open it from there, and choose this again.") + return + } + let command = "mkdir -p /usr/local/bin && ln -sf \(shellQuote(tool)) \(target)" + let source = "do shell script \"\(appleScriptEscape(command))\" with administrator privileges" + var error: NSDictionary? + _ = NSAppleScript(source: source)?.executeAndReturnError(&error) + if let error { + if (error[NSAppleScript.errorNumber] as? Int) == -128 { return } // the user cancelled + fallback(command) + return + } + inform("jev-serve is installed", + "Run it in Terminal:\n\n jev-serve --help\n jev-serve --config jev-serve.json\n\nIt's a link to the copy inside JevDK, so it updates when JevDK does. Export a config with File → Export Server Config….") + } + + /// The same, for the user to run: shown with a Copy button. + static func fallback(_ command: String) { + let alert = NSAlert() + alert.messageText = "Install jev-serve from Terminal" + alert.informativeText = "Run this in Terminal (it asks for your password):\n\nsudo sh -c '\(command.replacingOccurrences(of: "'", with: "'\\''"))'" + alert.addButton(withTitle: "Copy Command") + alert.addButton(withTitle: "Close") + if alert.runModal() == .alertFirstButtonReturn { + NSPasteboard.general.clearContents() + NSPasteboard.general.setString("sudo sh -c '\(command.replacingOccurrences(of: "'", with: "'\\''"))'", forType: .string) + } + } + + static func inform(_ title: String, _ text: String) { + let alert = NSAlert() + alert.messageText = title + alert.informativeText = text + alert.runModal() + } + + /// 'path' with single quotes escaped, for sh. + static func shellQuote(_ s: String) -> String { "'" + s.replacingOccurrences(of: "'", with: "'\\''") + "'" } + + /// The text as the inside of an AppleScript string literal. + static func appleScriptEscape(_ s: String) -> String { + s.replacingOccurrences(of: "\\", with: "\\\\").replacingOccurrences(of: "\"", with: "\\\"") + } +} + /// JevDK → About: the icon, the version, and where to read about decision models. enum AboutPanel { static let webPage = URL(string: "https://thisbrain.ai/locallm/jev.html")! diff --git a/examples/jevdk/scripts/release_macos.sh b/examples/jevdk/scripts/release_macos.sh index 6026574..55dfe02 100755 --- a/examples/jevdk/scripts/release_macos.sh +++ b/examples/jevdk/scripts/release_macos.sh @@ -1,7 +1,8 @@ #!/usr/bin/env bash set -euo pipefail -# Builds a Developer ID–signed, notarized, stapled JevDK.app and a drag-to-Applications DMG. +# Builds a Developer ID–signed, notarized, stapled JevDK.app (with jev-serve inside, from +# ../jev-serve) and a drag-to-Applications DMG. # # VERSION=0.1.0 \ # APP_IDENTITY="Developer ID Application: Your Name (TEAMID)" \ @@ -171,6 +172,27 @@ else echo "warning: no NOTICE found next to this repository; the app ships without it" >&2 fi +# jev-serve rides inside the app (Contents/MacOS/jev-serve) and uses the app's frameworks, so it's +# signed and notarized with the app and covered by the app's stapled ticket: a browser download +# runs without Gatekeeper prompts (a loose framework next to a CLI can't carry a ticket). +# JevDK → Install jev-serve Command… links it onto the PATH. +JEV_SERVE_ROOT="$APP_ROOT/../jev-serve" +if [[ -d "$JEV_SERVE_ROOT" ]]; then + echo "Building jev-serve (release, arm64) to embed..." + JS_VERSION_FILE="$JEV_SERVE_ROOT/Sources/JevServe/Version.swift" + cp "$JS_VERSION_FILE" "$BUILD_DIR/Version.swift.orig" + trap 'cp "$BUILD_DIR/Version.swift.orig" "$JS_VERSION_FILE" 2>/dev/null || true' EXIT + sed -i '' "s/^let jevServeVersion = .*/let jevServeVersion = \"$VERSION\"/" "$JS_VERSION_FILE" + ( cd "$JEV_SERVE_ROOT" && swift build -c release --arch arm64 --product jev-serve ) + JS_BIN="$(cd "$JEV_SERVE_ROOT" && swift build -c release --arch arm64 --show-bin-path)/jev-serve" + cp "$BUILD_DIR/Version.swift.orig" "$JS_VERSION_FILE" + cp "$JS_BIN" "$CONTENTS_DIR/MacOS/jev-serve" + # Its frameworks are the app's, in Contents/Frameworks. (Before signing: this edits the binary.) + install_name_tool -add_rpath @executable_path/../Frameworks "$CONTENTS_DIR/MacOS/jev-serve" +else + echo "warning: $JEV_SERVE_ROOT not found; the app ships without jev-serve" >&2 +fi + # The published xcframework zips carry AppleDouble/xattr detritus that breaks signing. find "$APP_DIR" -name '._*' -delete xattr -cr "$APP_DIR" @@ -191,6 +213,7 @@ for fw in ${FRAMEWORKS[@]+"${FRAMEWORKS[@]}"}; do # (empty-array safe under set sign "$fw/$(basename "$fw" .framework)" sign "$fw" done +if [[ -f "$CONTENTS_DIR/MacOS/jev-serve" ]]; then sign "$CONTENTS_DIR/MacOS/jev-serve"; fi sign "$APP_DIR" codesign --verify --deep --strict --verbose=2 "$APP_DIR" From baa919e44d8fada98abf3341e6f2afedbac3eb9f Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 08:53:03 -0700 Subject: [PATCH 56/73] annotated-examples: jev-serve added; jevdk brought up to date - jevdk: 2,819 / 3,080 lines, 15 files. Excerpts regenerated from the current source (two had drifted): model versions in the model list, Export Questions with its confirmation, Export Server Config, Open reading an exported set; intro covers MLXModelProvider's versions and the app carrying jev-serve. Tally 21 Core, 9 Inference, 8 Remote. - jev-serve (new): 591 / 640 lines, 4 files, 13 tests. Excerpts: the decision route (JevWire decode, limits, decideWithDiagnostics, encode) and the Decider protocol from Service.swift; model readiness (pinned MLXModelProvider, availability, validate, download) and the providers from main.swift. Tally 16 Core, 15 Inference. - Summary table rows for both; jev-serve in the legend. Every excerpt checked as an exact contiguous copy of the source. Co-Authored-By: Claude Opus 5.5 --- docs/annotated-examples.md | 292 ++++++++++++++++++++++++++++++++++--- 1 file changed, 270 insertions(+), 22 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index 281a75b..e950a96 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -2,7 +2,7 @@ The full source of every reference app, with every line that actually touches the SDK marked `// ← SDK` (Core), `// ← SDK (Inference)` (the MLX runtime — `code-buddy`, `repo-qa-local`, -`workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, `mlx-control-room`, `mcp-chat` and `jevdk`), `// ← SDK (Remote)` (online providers — +`workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, `mlx-control-room`, `mcp-chat`, `jevdk` and `jev-serve`), `// ← SDK (Remote)` (online providers — `model-switch`, `security-demo` and `jevdk`), `// ← SDK (MCPAppsHost)` (showing MCP servers' interactive views — `mcp-chat`), or `// ← Components` (`components-demo`, `components-updates-demo`, and `mcp-chat`). Everything else is ordinary SwiftUI/Foundation — the point of marking it this way is to make obvious just how little of each file is SDK-specific plumbing. @@ -41,7 +41,8 @@ non-comment lines; these examples are commented far more heavily than production | [`aiql`](#examplesaiql) | 402 | 459 | a plain-English request → one read-only SQL `SELECT` over an MCP dataset → the CSV you asked for, sandboxed SwiftUI, zero fabricated values; a pinned default model and a trust policy for the free-text picker | | [`vistanova`](#examplesvistanova) | 931 | 1,294 | a tiny local search engine: web search through a Tavily MCP server on one local model, summaries from a **pinned** MLX model on another; defends against a model that skips the tool call (7 files) | | [`mcp-chat`](#examplesmcp-chat) | 938 | 1,026 | a chat with a local model where an MCP server's own interactive view (Todoist's task list) shows up under the tool call: the conversation is `hostTranscript`, the view is `MCPAppsHost` (4 files, views excerpted) | -| [`jevdk`](#examplesjevdk) | 2,627 | 2,862 | a playground for decision-model questions: local OpenJev and hosted Jev side by side through one API, marked answers, calibration, and the tested questions exported for an app (14 files, excerpted) | +| [`jevdk`](#examplesjevdk) | 2,819 | 3,080 | a playground for decision-model questions: local OpenJev and hosted Jev side by side through one API, marked answers, calibration, and the tested setup exported for an app or for jev-serve (15 files, excerpted) | +| [`jev-serve`](#examplesjev-serve) | 591 | 640 | hosted Jev's HTTP API answered by OpenJev on this Mac: `JevWire` for the wire format, the tested setup from JevDK's config, models pinned and downloaded (4 files, excerpted) | | [`components-updates-demo`](#examplescomponents-updates-demo) | 151 | 170 | the `Components` model **onboarding**, **update** and **versions** views, driven by simulated sources so every state is reachable | | [`mlx-control-room`](#examplesmlx-control-room) | 1,434 | 1,812 | every MLX knob with a gauge, plus the supply-chain flow made visible: validate, download, **pin**, update, roll back, clean up (excerpted; UI omitted) | @@ -3783,16 +3784,16 @@ because a curated adapter lives in a different namespace. ## `examples/jevdk` -*2,627 lines of code across 14 files (2,862 with comments). Excerpted: the SDK-facing parts of +*2,819 lines of code across 15 files (3,080 with comments). Excerpted: the SDK-facing parts of `AppModel.swift` and `HostedBackends.swift` are below; the editor, batch grid, calibration -sheet, model browser and results-CSV writer are plain SwiftUI and Foundation and are omitted -(`// …`).* +sheet, model browser, About panel and results-CSV writer are plain SwiftUI and Foundation and are +omitted (`// …`).* A playground for **decision models** ([`sdk-guide.md` §6c](sdk-guide.md#6c-decision-models-jev--labdecide); for what a decision model is, see [Decision models (Jev) in your app](https://thisbrain.ai/locallm/jev.html)). Write yes / no, choice and score questions, run them on a local MLX model and on hosted Jev side -by side, mark the right answers, calibrate, and export the tested questions for your app. Three -SDK pieces carry it: +by side, mark the right answers, calibrate, and export the tested setup for your app or for +[`jev-serve`](#examplesjev-serve). Four SDK pieces carry it: - **`lab.decide`** for every hosted run: each enabled backend is a `JevDecisionProvider` registered under its own scheme with its own decision route, so the demo and the keyed @@ -3802,7 +3803,14 @@ SDK pieces carry it: and how much probability fell on the allowed labels. It validates against the provider's limits first, as `lab.decide` would. - **`DecisionCalibration`, `DecisionQuestionSet`, `DecisionAnswerSet`** for the workflow around - the runs: fitting, exporting for an app, and importing / exporting marked answers as CSV. + the runs: fitting, exporting the questions with their `Tuning` (model, version, wrapper, + calibration), opening an exported file again, and importing / exporting marked answers as CSV. +- **`MLXModelProvider`** lists, checks and downloads models, and reports each one's version + (`resolvedRevision`), which every export records so what ships is what was tested. + +**Export Server Config…** writes the same `Tuning` into a `jev-serve` config (`JevServeConfig`, +the example's own type in `OpenJevKit`). The released JevDK app carries `jev-serve` inside it; the +app menu's **Install jev-serve Command…** links it onto the PATH (that's plain AppKit, not shown). Links `LocalLMLabSDKCore`, `LocalLMLabSDKInference` and `LocalLMLabSDKRemote`. Not sandboxed; API keys live in the Keychain (or, for headless runs, environment variables), never in a file. @@ -3810,9 +3818,10 @@ API keys live in the Keychain (or, for headless runs, environment variables), ne ### `Sources/JevDK/AppModel.swift` (excerpt) Runs one input on every enabled backend concurrently: hosted through `lab.decide`, local through -the OpenJev provider directly. Then the files: the questions exported for an app, answers imported -from a CSV, and Open, which also reads an exported `.decisions.json` back. Note there is no prompt building and no answer parsing anywhere: -questions go in typed, answers come back typed. +the OpenJev provider directly. Then the files: the questions exported for an app (and for +jev-serve), answers imported from a CSV, and Open, which also reads an exported `.decisions.json` +back. There's no prompt building and no answer parsing anywhere: questions go in typed, answers +come back typed. ```swift import AppKit @@ -3824,16 +3833,17 @@ import Observation import OpenJevKit import UniformTypeIdentifiers // … + /// The SDK front door hosted runs go through. let lab = LocalLMLab() // ← SDK /// Local models: listing, preflight, download, and the weights OpenJev scores on. let mlx = MLXModelProvider() // ← SDK (Inference) - private var batchTask: Task? // … func refreshModels() { let probe = OpenJevDecisionProvider(mlx: mlx) // ← SDK (Inference) models = mlx.installed.map { m in // ← SDK (Inference) LocalChoice(repoID: m.repoID, sizeBytes: m.sizeBytes, - isMoE: probe.warnings(for: Self.openJevID(m.repoID)).contains(.mixtureOfExperts)) // ← SDK (Inference) + isMoE: probe.warnings(for: Self.openJevID(m.repoID)).contains(.mixtureOfExperts), // ← SDK (Inference) + revision: m.resolvedRevision) }.sorted { ($0.sizeBytes ?? 0) < ($1.sizeBytes ?? 0) } // … private struct HostedPlan { @@ -3918,6 +3928,7 @@ import UniformTypeIdentifiers for await r in group { out.append(r) } return out.sorted { $0.0 < $1.0 }.map(\.1) } + } // … /// Fit on the marked answers with the SDK's `DecisionCalibration.fit`. func fitCalibration() { @@ -3933,8 +3944,6 @@ import UniformTypeIdentifiers applyCalibration = true } // … - // MARK: Question set for the app, answer sets - /// What the questions are being tested with, for the exported question set: the local model, /// its revision, the wrapper and (when it matches) the calibration. Nil with Local off. var exportTuning: DecisionQuestionSet.Tuning? { // ← SDK @@ -3955,17 +3964,69 @@ import UniformTypeIdentifiers panel.allowedContentTypes = [.json] let base = set.name.isEmpty ? "questions" : set.name.lowercased().replacingOccurrences(of: " ", with: "-") panel.nameFieldStringValue = "\(base).decisions.json" - panel.message = "A question set for your app: bundle it and load it with DecisionQuestionSet(contentsOf:)." + panel.message = "A question set for your app: bundle it and load it with DecisionQuestionSet(contentsOf:)." // ← SDK guard panel.runModal() == .OK, let url = panel.url else { return } do { - let file = DecisionQuestionSet(name: set.name, questions: set.questions.map(\.sdkQuestion), tuning: exportTuning) // ← SDK - try file.jsonData().write(to: url) // ← SDK + let tuning = exportTuning + let file = DecisionQuestionSet(name: set.name, questions: set.questions.map(\.sdkQuestion), tuning: tuning) // ← SDK + try file.jsonData().write(to: url) errorMessage = nil + notice = "Exported \(set.questions.count) questions to \(url.lastPathComponent)" + Self.tuningSummary(tuning) + "." } catch { errorMessage = "Couldn't export: \(error.localizedDescription)" } } + /// ", for Qwen3-4B-4bit pinned to 4dcb3d1, calibrated" — what an export carries. + static func tuningSummary(_ t: DecisionQuestionSet.Tuning?) -> String { // ← SDK + guard let t, let model = t.model else { return " (no local model: questions only)" } + let name = model.rest.split(separator: "/").last.map(String.init) ?? model.rest // ← SDK + var s = ", for \(name)" + if let r = t.revision { s += " pinned to \(r.prefix(7))" } + s += t.calibration == nil ? ", not calibrated" : ", calibrated" + return s + } + + /// Add this setup (local model, its version, system instructions, calibration) to a jev-serve + /// config, creating one if needed (see the jev-serve example). An existing config keeps its other + /// models, listen address and token; the entry with this question set's name is replaced. + func exportServerConfig() { + guard let tuning = exportTuning else { + errorMessage = "Pick a local model first: jev-serve serves a local model." + return + } + let panel = NSSavePanel() + panel.allowedContentTypes = [.json] + panel.nameFieldStringValue = "jev-serve.json" + panel.message = "A jev-serve config. If the file exists, this setup is added to it (or replaces the one with the same name)." + guard panel.runModal() == .OK, let url = panel.url else { return } + var config: JevServeConfig + var created = false + if FileManager.default.fileExists(atPath: url.path) { + do { config = try JevServeConfig.read(url) } catch { + errorMessage = "\(url.lastPathComponent) isn't a jev-serve config, so it wasn't changed: \(error.localizedDescription)" + return + } + } else { + created = true + let alert = NSAlert() + alert.messageText = "Require a token?" + alert.informativeText = "With a token, jev-serve only answers requests that send it (Authorization: Bearer …). It's saved in the config file, which only you can read." + alert.addButton(withTitle: "Generate a Token") + alert.addButton(withTitle: "No Token") + config = JevServeConfig(token: alert.runModal() == .alertFirstButtonReturn ? JevServeConfig.newToken() : nil) + } + let name = Self.slug(set.name.isEmpty ? "decider" : set.name) + config.upsert(.init(name: name, tuning: tuning)) + do { + try config.write(to: url) + errorMessage = nil + notice = "\(created ? "Wrote" : "Updated") \(url.lastPathComponent): “\(name)”" + Self.tuningSummary(tuning) + + (config.token != nil ? ", token required" : "") + ". Serve it with: jev-serve --config \(url.lastPathComponent)" + } catch { + errorMessage = "Couldn't write \(url.lastPathComponent): \(error.localizedDescription)" + } + } // … /// Load an answer set (CSV: `input`, then a column per question with the correct answer) into /// the batch list and the marks. Inputs with line breaks are joined onto one line, since the @@ -4084,13 +4145,200 @@ enum APIKeys { static func account(_ backend: HostedBackend) -> String { "key.\(backend.rawValue)" } } +``` + +**Tally:** 21 lines Core (`// ← SDK`), 9 Inference, 8 Remote, in the excerpts above. The +decision API itself is a handful of calls (`register`/`replace(decision:)`, `route(decision:)`, +`decide`, `decideWithDiagnostics`, `DecisionCalibration.fit`, `DecisionQuestionSet`, +`DecisionAnswerSet`); the rest of the app is the playground around them. + +## `examples/jev-serve` + +*591 lines of code across 4 files (640 with comments), plus 13 tests. Excerpted: the routes' +decision path and the decider from `Service.swift`, and model readiness and the providers from +`main.swift`; the HTTP/1.1 server (`HTTPServer.swift`, on `Network.framework`), the health and +model-list routes, the token check, the queue, argument parsing and `--self-check` are omitted +(`// …`).* + +Hosted Jev's HTTP API, answered on this Mac: `POST /api/alpha/decisions` (OpenRouter / TypeSafe) +and `POST /v1/classifier` (Featherless) take hosted Jev's request and return its response, from +OpenJev on a local MLX model. Code that calls hosted Jev, in any language, switches by changing its +base URL ([`sdk-guide.md` §6c](sdk-guide.md#serving-decisions-over-http-jev-serve)). It serves the +setup tested in [`jevdk`](#examplesjevdk), from the config JevDK exports. Three SDK pieces carry it: + +- **`JevWire`** (Core) reads the request and writes the response, keeping questions and options in + the caller's order: the same code the SDK's `JevDecisionProvider` uses as a client, so the + client and this server can't disagree (the tests check it both ways). +- **`OpenJevDecisionProvider(mlx:tunedWith:)`** answers with each config entry's wrapper and + calibration; `tuningMismatch(for:)` turns calibration off, with a warning, if the model on disk + isn't the version it was fitted on. +- **`MLXModelProvider(pinnedRevisions:)`** makes each model ready before the server listens: + pinned to the version in the config, then found, verified or pre-flighted and downloaded. + +The server talks to its model through a small `Decider` protocol, so the routes, token, limits and +queue are tested with a scripted decider and no GPU. Links `LocalLMLabSDKCore` and +`LocalLMLabSDKInference` (tests add `LocalLMLabSDKRemote` for the client round trip). The JevDK +download carries it; `swift build` builds it. + +### `Sources/JevServeKit/Service.swift` (excerpt) + +A decision request: decode, pick the model, validate against the decider's limits (plus the +config's input cap), wait its turn, decide, encode. Every failure is hosted Jev's error shape. + +```swift +/// What answers a served model's questions. jev-serve uses `OpenJevDecider` (OpenJev on MLX); tests +/// use a scripted one, so routing, auth, limits and the queue are tested without a GPU. +public protocol Decider: Sendable { + var limits: DecisionLimits { get } // ← SDK + var calibrated: Bool { get } + /// The decision, and the input tokens it read (for `usage`). + func decide(_ request: DecisionRequest) async throws -> (decision: Decision, inputTokens: Int?) // ← SDK +} + // … +/// OpenJev on a local MLX model, with one config entry's wrapper and calibration. +public struct OpenJevDecider: Decider { + public let provider: OpenJevDecisionProvider // ← SDK (Inference) + public let id: ModelID // ← SDK + + public init(provider: OpenJevDecisionProvider, id: ModelID) { // ← SDK (Inference) + self.provider = provider + self.id = id + } + + public var limits: DecisionLimits { provider.limits } // ← SDK + public var calibrated: Bool { provider.calibration != nil } + + public func decide(_ request: DecisionRequest) async throws -> (decision: Decision, inputTokens: Int?) { // ← SDK + let (decision, diagnostics) = try await provider.decideWithDiagnostics(request, using: id) // ← SDK (Inference) + return (decision, diagnostics.sharedPrefixTokens) + } +} + // … + private func decide(_ req: HTTPRequest) async -> (HTTPResponse, String) { + let wire: JevWire.Request // ← SDK + do { wire = try JevWire.decodeRequest(req.body) } catch { // ← SDK + return (Self.error(400, Self.message(error)), "") + } + let name = wire.model ?? defaultModel + guard let served = model(named: name) else { + return (Self.error(404, "jev-serve doesn't serve “\(name)”. Served: \(models.map(\.name).joined(separator: ", ")).", type: "not_found"), "") + } + let note = "\(served.name) \(wire.request.questions.count)q" + let l = served.decider.limits + let limits = DecisionLimits(maxQuestions: l.maxQuestions, maxChoiceCriteria: l.maxChoiceCriteria, // ← SDK + maxScoreLevels: l.maxScoreLevels, maxStateCharacters: maxStateCharacters ?? l.maxStateCharacters) + do { try wire.request.validate(against: limits) } catch { // ← SDK + return (Self.error(400, Self.message(error)), note) + } + guard await queue.acquire() else { + return (Self.error(503, "Busy: too many requests waiting. Try again shortly.", type: "overloaded"), note) + } + do { + var (decision, tokens) = try await served.decider.decide(wire.request) + await queue.release() + decision.usage = DecisionUsage(inputTokens: tokens, outputTokens: 0, cost: 0) // ← SDK + let body = try JevWire.encodeResponse(decision, model: served.repo, request: wire.request) // ← SDK + return (.json(200, body), note) + } catch let e as DecisionError { // ← SDK + await queue.release() + switch e { + case .invalidRequest: return (Self.error(400, Self.message(e)), note) + case .unavailable: return (Self.error(503, Self.message(e), type: "unavailable"), note) + default: return (Self.error(500, Self.message(e), type: "server_error"), note) + } + } catch { + await queue.release() + return (Self.error(500, Self.message(error), type: "server_error"), note) + } + } +``` + +### `Sources/JevServe/main.swift` (excerpt) +Startup: one MLX provider pinned to the configured versions; each model made ready (with download +progress); each config entry's provider built from its `Tuning`; the default model loaded before +the server listens. + +```swift +let mlx = MLXModelProvider(residentModelLimit: max(1, repos.count), pinnedRevisions: pins, pinStore: MLXFilePinStore()) // ← SDK (Inference) +// … +@MainActor func ensureReady(_ repo: String) async { + let id = ModelID(scheme: "mlx", rest: repo)! // ← SDK + switch mlx.availability(for: id) { // ← SDK (Inference) + case .available: + return + case .unavailable(_, let detail): + fail("\(repo) can't run here: \(detail)") + case .needsCredential: + fail("\(repo) needs a Hugging Face credential") + case .notDownloaded: + break + @unknown default: + break + } + let other = mlx.installed.first { $0.repoID == repo }?.resolvedRevision // ← SDK (Inference) + let what: String + if let want = pins[repo], let other, other != want { + what = "\(repo) version \(want.prefix(7)) (the one in the config) isn't on this Mac; version \(other.prefix(7)) is" + } else { + what = "\(repo) isn't on this Mac" + } + if noDownload { fail("\(what), and --no-download is set. Drop --no-download to fetch it, or download it in JevDK's Models.") } + if other != nil { jevServeLog("\(what): fetching the configured version") } + if let pre = try? await mlx.validate(repo), !pre.passed { // ← SDK (Inference) + fail("pre-flight failed for \(repo) (\(pre.failedStage?.rawValue ?? "?")): \(pre.detail ?? "")") + } + let version = pins[repo].map { " @ \($0.prefix(7))" } ?? "" + jevServeLog("downloading \(repo)\(version) (a copy already in the Hugging Face cache is verified, only missing files fetched)…") + do { + var last = -1 + for try await event in mlx.download(repo) { // ← SDK (Inference) + if case .progress(_, _, let f) = event { + let pct = Int(f * 100) + if pct != last { + last = pct + FileHandle.standardError.write(Data("\u{1B}[2K\r \(repo)\(version) \(pct)%".utf8)) + } + } + } + FileHandle.standardError.write(Data("\u{1B}[2K\r".utf8)) + jevServeLog("\(repo): ready") + } catch { + FileHandle.standardError.write(Data("\n".utf8)) + fail("download of \(repo) failed: \(error.localizedDescription)") + } +} +// … +var served: [ServedModel] = [] +for m in config.models { + let repo = m.tuning.model!.rest // ← SDK + var tuning = m.tuning + let set = DecisionQuestionSet(name: m.name, questions: [], tuning: tuning) // ← SDK + let probe = OpenJevDecisionProvider(mlx: mlx, tunedWith: set) // ← SDK (Inference) + let id = ModelID(scheme: probe.scheme, rest: repo)! // ← SDK + if let why = probe.tuningMismatch(for: id) { // ← SDK (Inference) + jevServeLog("warning: “\(m.name)”: \(why); serving it without its calibration") + tuning.calibration = nil + } + if probe.warnings(for: id).contains(.mixtureOfExperts) { // ← SDK (Inference) + jevServeLog("warning: \(repo) is a mixture-of-experts model; its probabilities are unstable as a decider") + } + let provider = OpenJevDecisionProvider(mlx: mlx, tunedWith: DecisionQuestionSet(name: m.name, questions: [], tuning: tuning)) // ← SDK (Inference) + let revision = mlx.effectivePin(for: repo)?.revision ?? tuning.revision // ← SDK (Inference) + ?? mlx.installed.first { $0.repoID == repo }?.resolvedRevision // ← SDK (Inference) + served.append(ServedModel(name: m.name, repo: repo, revision: revision, decider: OpenJevDecider(provider: provider, id: id))) +} +let defaultName = config.defaultModel ?? served[0].name +guard let defaultModel = served.first(where: { $0.name == defaultName }) else { + fail("defaultModel “\(defaultName)” isn't one of the models") +} +jevServeLog("loading \(defaultModel.repo)…") +if let d = defaultModel.decider as? OpenJevDecider { await d.provider.prewarm(d.id) } // ← SDK (Inference) ``` -**Tally:** 19 lines Core (`// ← SDK`), 9 Inference, 8 Remote, in the excerpts -above. The decision API itself is a handful of calls (`register`/`replace(decision:)`, -`route(decision:)`, `decide`, `decideWithDiagnostics`, `DecisionCalibration.fit`, -`DecisionQuestionSet`, `DecisionAnswerSet`); the rest of the app is the playground around them. +**Tally:** 16 lines Core (`// ← SDK`), 15 Inference, 0 Remote, in the excerpts above. The wire +format is three `JevWire` calls; the decision is one `decideWithDiagnostics`; the rest is +HTTP, configuration and model housekeeping. ## `examples/mcp-chat` From 330198f559214abd5730d08abb46faf54f649d2c Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 08:56:33 -0700 Subject: [PATCH 57/73] README: JevDK download from ancientcomputing/jevdk; jev-serve comes with it Co-Authored-By: Claude Opus 5.5 --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5417b08..96d1cde 100644 --- a/README.md +++ b/README.md @@ -174,11 +174,12 @@ Roughly simplest to fullest — every one runnable, with full annotated source i questions: run them on a local MLX model and on hosted Jev side by side, score a batch against your marked answers, calibrate, compare models, and export the tested questions for your app. [Developer's guide](https://thisbrain.ai/locallm/jdk-guide.html); - [download the app](https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/JevDK-0.1.0-arm64.dmg) - (macOS 27, signed and notarized, a preview built on SDK 2.0.0-dev). + [download the app](https://github.com/ancientcomputing/jevdk/releases/latest) from + [ancientcomputing/jevdk](https://github.com/ancientcomputing/jevdk) (macOS 27, signed and notarized; includes jev-serve). - **[examples/jev-serve/](examples/jev-serve/)** (2.0) — hosted Jev's HTTP API (OpenRouter's and Featherless's) answered by OpenJev on your Mac: point existing hosted-Jev code, in any language, at it to get private, on-device decisions, using the setup you tested and exported from JevDK. + It comes with the JevDK download (**JevDK → Install jev-serve Command…**). Plus **[examples/api-lab/](examples/api-lab/)** (scripts + a chat app for the OpenAI-compatible endpoint) and **[examples/localai-cli/](examples/localai-cli/)** / **[localai-cli-swift/](examples/localai-cli-swift/)** From 739359769442696a129e4e257c743e9e58dc0f26 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 08:59:37 -0700 Subject: [PATCH 58/73] =?UTF-8?q?sync-jevdk-repo:=20keep=20the=20jevdk=20r?= =?UTF-8?q?epo's=20own=20SDK=20version=20(SDK=5FVERSION=3D=E2=80=A6=20to?= =?UTF-8?q?=20change=20it)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- scripts/sync-jevdk-repo.sh | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/scripts/sync-jevdk-repo.sh b/scripts/sync-jevdk-repo.sh index 6c9b666..c8fa00a 100755 --- a/scripts/sync-jevdk-repo.sh +++ b/scripts/sync-jevdk-repo.sh @@ -5,12 +5,19 @@ set -euo pipefail # ancientcomputing/jevdk, the standalone repo end users download them from. # # scripts/sync-jevdk-repo.sh ../jevdk +# SDK_VERSION=2.0.0 scripts/sync-jevdk-repo.sh ../jevdk # also move the clone to another SDK # # One-way: this repo (examples/jevdk, examples/jev-serve) is the source of truth, and both folders # are copied unchanged into /jevdk and /jev-serve, so they build there exactly as # they do here (jev-serve takes the SDK binaries from ../jevdk). The clone's own files (README.md, # NOTICE, SOURCE.md, scripts/) are left alone, except NOTICE, which is refreshed from this repo's. # Then it builds both, runs jev-serve's tests, and shows what changed. It never commits or pushes. +# +# The SDK version is the one exception to "unchanged". examples/jevdk follows the SDK's releases +# closely; the jevdk repo stays on the main x.0.0 releases unless a newer one has something it +# needs. So the clone keeps its own `defaultSDKVersion` (jevdk/Package.swift), and only +# SDK_VERSION= changes it. The release table (knownSDKReleases) is copied as is, so it +# must list that version. cd "$(git rev-parse --show-toplevel)" SRC="$PWD" @@ -23,6 +30,10 @@ git -C "$DEST" remote get-url origin | grep -q "ancientcomputing/jevdk" \ echo "== Checking this repo for private references first" ./scripts/check-public-hygiene.sh +PKG=jevdk/Package.swift +VERSION_RE='^let defaultSDKVersion = "\([^"]*\)"' +KEEP="${SDK_VERSION:-$( [[ -f "$DEST/$PKG" ]] && sed -n "s/$VERSION_RE/\1/p" "$DEST/$PKG" )}" + EXCLUDES=(--exclude .build/ --exclude .swiftpm/ --exclude build/ --exclude dist/ --exclude Package.resolved --exclude xcuserdata/ --exclude .DS_Store --exclude jev-serve.json --exclude .env --exclude 'key.env') for d in jevdk jev-serve; do @@ -31,6 +42,13 @@ for d in jevdk jev-serve; do done cp "$SRC/NOTICE" "$DEST/NOTICE" +if [[ -n "$KEEP" ]]; then + grep -q "^ \"$KEEP\": SDKRelease(" "$DEST/$PKG" \ + || { echo "SDK $KEEP isn't in knownSDKReleases in examples/$PKG; add it there first" >&2; exit 1; } + sed -i '' "s/$VERSION_RE/let defaultSDKVersion = \"$KEEP\"/" "$DEST/$PKG" +fi +echo "== SDK: the jevdk repo builds against $(sed -n "s/$VERSION_RE/\1/p" "$DEST/$PKG") (examples: $(sed -n "s/$VERSION_RE/\1/p" "$SRC/examples/$PKG"))" + echo "== Building" ( cd "$DEST/jevdk" && swift build -c release 2>&1 | tail -1 ) ( cd "$DEST/jev-serve" && swift build -c release 2>&1 | tail -1 ) From 470135249624e2ac442f0f8d1a8b45fba78ce6c3 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 5 Oct 2026 14:33:35 -0700 Subject: [PATCH 59/73] JevDK: the app icon in the bundle (AppIcon.icns), so Finder and the DMG show it The icon is drawn in code and was only set on the running app's Dock tile. The release script now runs JevDK --write-iconset, makes AppIcon.icns with iconutil and sets CFBundleIconFile, before signing. Co-Authored-By: Claude Opus 5.5 --- examples/jevdk/Sources/JevDK/JevDKApp.swift | 33 +++++++++++++++++++++ examples/jevdk/scripts/release_macos.sh | 13 ++++++++ 2 files changed, 46 insertions(+) diff --git a/examples/jevdk/Sources/JevDK/JevDKApp.swift b/examples/jevdk/Sources/JevDK/JevDKApp.swift index f0be710..676aa5d 100644 --- a/examples/jevdk/Sources/JevDK/JevDKApp.swift +++ b/examples/jevdk/Sources/JevDK/JevDKApp.swift @@ -25,6 +25,13 @@ struct JevDKApp: App { model: args.count > i + 3 && !args[i + 3].hasPrefix("--") ? args[i + 3] : nil) } + // `JevDK --write-iconset .iconset` writes AppIcon as PNGs at every size `iconutil` + // wants, then exits: the release script makes the bundle's AppIcon.icns from them, so + // Finder and the DMG show the same icon as the Dock. + if let i = args.firstIndex(of: "--write-iconset"), args.count > i + 1 { + AppIcon.writeIconset(to: URL(fileURLWithPath: args[i + 1])) + } + // Launched with `swift run` there is no app bundle, so macOS starts this as a // background-style process that never becomes active. Claim regular-app status // (only if needed; re-setting it logs "Task policy set failed"). @@ -472,6 +479,32 @@ enum AboutPanel { enum AppIcon { static let blue = NSColor(srgbRed: 0x2f / 255, green: 0x6f / 255, blue: 0xed / 255, alpha: 1) + /// The `.iconset` folder `iconutil -c icns` turns into AppIcon.icns. Exits. + static func writeIconset(to dir: URL) -> Never { + do { + try FileManager.default.createDirectory(at: dir, withIntermediateDirectories: true) + for points in [16, 32, 128, 256, 512] { + for scale in [1, 2] { + let px = points * scale + guard let rep = NSBitmapImageRep(bitmapDataPlanes: nil, pixelsWide: px, pixelsHigh: px, + bitsPerSample: 8, samplesPerPixel: 4, hasAlpha: true, isPlanar: false, + colorSpaceName: .deviceRGB, bytesPerRow: 0, bitsPerPixel: 0), + let context = NSGraphicsContext(bitmapImageRep: rep) else { throw CocoaError(.fileWriteUnknown) } + NSGraphicsContext.saveGraphicsState() + NSGraphicsContext.current = context + image(size: CGFloat(px)).draw(in: NSRect(x: 0, y: 0, width: px, height: px)) + NSGraphicsContext.restoreGraphicsState() + let name = "icon_\(points)x\(points)\(scale == 2 ? "@2x" : "").png" + try rep.representation(using: .png, properties: [:])!.write(to: dir.appendingPathComponent(name)) + } + } + exit(0) + } catch { + FileHandle.standardError.write(Data("Couldn't write the iconset: \(error)\n".utf8)) + exit(1) + } + } + static func image(size: CGFloat = 512) -> NSImage { NSImage(size: NSSize(width: size, height: size), flipped: false) { rect in // Full-bleed rounded square, corner radius as in LocalLM Lab's icon (14 of 64). diff --git a/examples/jevdk/scripts/release_macos.sh b/examples/jevdk/scripts/release_macos.sh index 55dfe02..a131a97 100755 --- a/examples/jevdk/scripts/release_macos.sh +++ b/examples/jevdk/scripts/release_macos.sh @@ -107,6 +107,7 @@ require_command xattr require_command codesign require_command ditto require_command hdiutil +require_command iconutil require_command shasum require_command spctl require_command xcrun @@ -159,6 +160,18 @@ ditto "$ARCHIVE_PATH/Products/Applications/${APP_NAME}.app" "$APP_DIR" /usr/libexec/PlistBuddy -c "Set :CFBundleShortVersionString $VERSION" "$CONTENTS_DIR/Info.plist" /usr/libexec/PlistBuddy -c "Set :CFBundleVersion $VERSION" "$CONTENTS_DIR/Info.plist" +# The app icon is drawn in code (AppIcon), which only the running app's Dock sees. Render it into +# Contents/Resources/AppIcon.icns so Finder and the DMG show it too. (Before signing: the bundle's +# resources and Info.plist are sealed by the signature.) +echo "Rendering the app icon..." +ICONSET="$BUILD_DIR/AppIcon.iconset" +rm -rf "$ICONSET" +"$CONTENTS_DIR/MacOS/$APP_NAME" --write-iconset "$ICONSET" +mkdir -p "$CONTENTS_DIR/Resources" +iconutil -c icns -o "$CONTENTS_DIR/Resources/AppIcon.icns" "$ICONSET" +/usr/libexec/PlistBuddy -c "Delete :CFBundleIconFile" "$CONTENTS_DIR/Info.plist" 2>/dev/null || true +/usr/libexec/PlistBuddy -c "Add :CFBundleIconFile string AppIcon" "$CONTENTS_DIR/Info.plist" + # Apache-2.0 §4(d): carry the SDK's NOTICE (it bundles the MLX stack) inside the app. It's at the # repository root: one level up in ancientcomputing/jevdk, two in ancientcomputing/locallm. NOTICE_FILE="" From 3462e72775ecf700c48aac6bfecbf45837530bc6 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Wed, 7 Oct 2026 09:41:13 -0700 Subject: [PATCH 60/73] Cascade 2.0.0-GA SDK checksums knownSDKReleases in Components and the 15 SDK examples is now 1.0.0-GA (previous) + 2.0.0-GA; defaultSDKVersion = "2.0.0-GA"; 2.0.0-dev dropped (jevdk and mcp-chat list 2.0.0-GA only). Each module's checksum is from the v2.0.0-GA release's .sha256, verified against the downloaded zips (Core faa92a02, Inference fde8babb, Remote d54f02c9, Claude 7afe7d79). "Requires SDK 2.0.0-dev or later" comments (jevdk, mcp-chat, aiql, MCPAppsHost) and the examples READMEs now say 2.0.0-GA; examples/README.md's knownSDKReleases / .sha256 samples too. Built: Components, MCPAppsHost and every example with swift build (fresh scratch path), VistaNova with xcodebuild. check-public-hygiene passes. Co-Authored-By: Claude --- Components/Package.swift | 8 ++++---- MCPAppsHost/Package.swift | 2 +- examples/README.md | 16 ++++++++-------- examples/aiql/Package.swift | 14 +++++++------- examples/aiql/README.md | 2 +- examples/code-buddy/Package.swift | 12 ++++++------ examples/jevdk/Package.swift | 18 +++++++++--------- examples/jevdk/README.md | 2 +- examples/mcp-chat/Package.swift | 10 +++++----- examples/mcp-chat/README.md | 2 +- examples/mlx-control-room/Package.swift | 12 ++++++------ examples/model-switch/Package.swift | 8 ++++---- examples/os-matrix/Package.swift | 12 ++++++------ examples/plate-today-tools/Package.swift | 8 ++++---- examples/plate-today/Package.swift | 8 ++++---- examples/repo-qa-local/Package.swift | 12 ++++++------ examples/repo-qa/Package.swift | 8 ++++---- examples/security-demo/Package.swift | 8 ++++---- examples/vistanova/Package.swift | 12 ++++++------ examples/workspace-buddy-local/Package.swift | 12 ++++++------ examples/workspace-buddy/Package.swift | 8 ++++---- 21 files changed, 97 insertions(+), 97 deletions(-) diff --git a/Components/Package.swift b/Components/Package.swift index 2850fff..0099868 100644 --- a/Components/Package.swift +++ b/Components/Package.swift @@ -24,16 +24,16 @@ struct SDKRelease { // or edit `defaultSDKVersion`. For a release not listed, add its entry (URL follows the pattern // below; checksum is the `.sha256` next to the zip on that GitHub release). Keep this in step // with `examples/model-switch/Package.swift`, which consumes Core through this package. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), - "2.0.0-dev": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" + "2.0.0-GA": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" ), ] diff --git a/MCPAppsHost/Package.swift b/MCPAppsHost/Package.swift index 9758d48..983216f 100644 --- a/MCPAppsHost/Package.swift +++ b/MCPAppsHost/Package.swift @@ -9,7 +9,7 @@ import PackageDescription // Core comes from Components, which declares the LocalLMLabSDKCore binary target and re-vends it // (two packages declaring that binary target is a hard SwiftPM error). So this package has no // SDK version or checksum of its own: it builds against whatever Components' `defaultSDKVersion` -// / LOCALLM_SDK_VERSION selects. Requires SDK 2.0.0-dev or later. +// / LOCALLM_SDK_VERSION selects. Requires SDK 2.0.0-GA or later. // // What is here: metadata parsing (`_meta.ui`) and the `ui://` resource loader (mime check, size, // SHA-256); `MCPAppsBridge`, the JSON-RPC state machine between a widget and the host, with an diff --git a/examples/README.md b/examples/README.md index 984e76b..aa8e31d 100644 --- a/examples/README.md +++ b/examples/README.md @@ -59,7 +59,7 @@ advanced model-layer features (Private Cloud Compute, open-weight/MLX, Claude, o need it — but it isn't universal: [`os-matrix/`](os-matrix/) is built and tested to run unchanged on **macOS 26** too (see its README). -The examples on this `2.0.0-dev` branch build against SDK **`2.0.0-dev`** (a pre-release); the ones on `main` +The examples on this `2.0.0-GA` branch build against SDK **`2.0.0-GA`**; the ones on `main` build against the latest stable release. No environment variable is needed for either. ### What each example needs @@ -171,13 +171,13 @@ previous one. (`code-buddy/sample-workspace/Package.swift` is not one of these dependency-free fixture for the code-buddy walkthrough, not an SDK consumer.) ```swift -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease(url: "…/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2…"), - "2.0.0-dev": SDKRelease(url: "…/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "c741066f…"), + "2.0.0-GA": SDKRelease(url: "…/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + checksum: "faa92a02…"), ] ``` @@ -188,10 +188,10 @@ every module ships its own checksum (`Core`, `Inference`, `Remote`, and `Claude` distinct `.xcframework.zip.sha256`; don't reuse Core's for another module): ```bash -curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip.sha256 -# → c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb -curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKClaude-2.0.0-dev.xcframework.zip.sha256 -# → 65ec39fe1f2965041e05793e86850f624d1f8bfd6ebbd8111b9c06f68c1975f2 +curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip.sha256 +# → faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029 +curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip.sha256 +# → 7afe7d79c63b67cc695d53a1378d5eee23fd508d2409908a717085622b86a771 ``` Then point `defaultSDKVersion` at it (works everywhere, Xcode included) or pass diff --git a/examples/aiql/Package.swift b/examples/aiql/Package.swift index 5158ec7..b339922 100644 --- a/examples/aiql/Package.swift +++ b/examples/aiql/Package.swift @@ -9,7 +9,7 @@ import PackageDescription // payload never enters the model's context), then project / filter / sort with the Core data // verbs (docs/sdk-guide.md §8b). The row data never passes through the model, so it can't be // fabricated. The data tools come from lab.mcp, file-backed (`setFileBackedOutput`, new in -// 2.0.0) — so this example needs SDK 2.0.0-dev or later. +// 2.0.0) — so this example needs SDK 2.0.0-GA or later. // // Combines three existing examples: workspace-buddy-local (SwiftUI + App Sandbox + MLX model + // folder picker), plate-today (MCP client + OAuth redirect wiring), repo-qa (tools from a live @@ -31,7 +31,7 @@ struct SDKRelease { // uses. `knownSDKReleases` carries this plus the previous release. Build against another // published version: set LOCALLM_SDK_VERSION in your shell (works for `swift build` / CI, NOT // inside Xcode), or edit `defaultSDKVersion` here. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( @@ -40,11 +40,11 @@ let knownSDKReleases: [String: SDKRelease] = [ inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/aiql/README.md b/examples/aiql/README.md index 0b4237a..5786287 100644 --- a/examples/aiql/README.md +++ b/examples/aiql/README.md @@ -103,7 +103,7 @@ export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer ``` Skip it if `xcode-select -p` already points at Xcode 27 (adjust the path if yours lives elsewhere). Leaves your system default alone; lasts only for the current terminal (re-run it in each new one, -or add it to your `~/.zshrc`). `Package.swift` builds against SDK `2.0.0-dev` with no further +or add it to your `~/.zshrc`). `Package.swift` builds against SDK `2.0.0-GA` with no further setup (AIQL needs it: it uses `lab.mcp.setFileBackedOutput`, new in 2.0) — it links **two** binaries, `LocalLMLabSDKCore.xcframework` and `LocalLMLabSDKInference.xcframework` (the MLX runtime), from that one GitHub Release. `export LOCALLM_SDK_VERSION=` to pin a different published release. **No Metal Toolchain diff --git a/examples/code-buddy/Package.swift b/examples/code-buddy/Package.swift index a85cfef..35cfd43 100644 --- a/examples/code-buddy/Package.swift +++ b/examples/code-buddy/Package.swift @@ -25,7 +25,7 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( @@ -34,11 +34,11 @@ let knownSDKReleases: [String: SDKRelease] = [ inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/jevdk/Package.swift b/examples/jevdk/Package.swift index 403a181..4c4b388 100644 --- a/examples/jevdk/Package.swift +++ b/examples/jevdk/Package.swift @@ -16,7 +16,7 @@ import PackageDescription // so it declares Core itself. Same SDKRelease/knownSDKReleases/failManifest version gate as every // other example here. // -// Requires SDK 2.0.0-dev or later (the decision API is new in 2.0), macOS 27 + Xcode 27. No Metal +// Requires SDK 2.0.0-GA or later (the decision API is new in 2.0), macOS 27 + Xcode 27. No Metal // Toolchain needed — the prebuilt Inference xcframework bundles the compiled default.metallib. struct SDKRelease { @@ -33,16 +33,16 @@ struct SDKRelease { // or edit `defaultSDKVersion` here. For a release not listed, add its entry (URL + the `.sha256` // next to each zip on the GitHub release) or just replace the strings in place. Only 2.0 // releases: this app does not build against 1.0. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "a48098d692084ccde5db7c9e2e39806b579b387147a36546995e606baa51099e", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", + remoteChecksum: "d54f02c93266d0dd34ea06075af78dd62de353f326ce5e850ff8b8cda5900d8b", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/jevdk/README.md b/examples/jevdk/README.md index 07af656..e27a5b1 100644 --- a/examples/jevdk/README.md +++ b/examples/jevdk/README.md @@ -9,7 +9,7 @@ swift build -c release ``` Needs an Apple-silicon Mac, macOS 27 and Xcode 27 (Swift 6.4). `Package.swift` builds against SDK -`2.0.0-dev` with no further setup (the decision API is new in 2.0): `LocalLMLabSDKCore`, +`2.0.0-GA` with no further setup (the decision API is new in 2.0): `LocalLMLabSDKCore`, `LocalLMLabSDKRemote` and `LocalLMLabSDKInference` (the MLX runtime) come from that GitHub Release. No Metal Toolchain needed; the prebuilt Inference xcframework bundles the compiled shaders. To build against another release, set `LOCALLM_SDK_VERSION` (see [the README one folder up](../README.md)). diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift index c0b61c2..8987e82 100644 --- a/examples/mcp-chat/Package.swift +++ b/examples/mcp-chat/Package.swift @@ -16,7 +16,7 @@ import PackageDescription // error. Components re-vends Core as a `.library` product for exactly this. Same // SDKRelease/knownSDKReleases/failManifest version gate as every other example here. // -// Requires SDK 2.0.0-dev or later (MCP Apps support, per-server trust and approval are new in 2.0), +// Requires SDK 2.0.0-GA or later (MCP Apps support, per-server trust and approval are new in 2.0), // macOS 27 + Xcode 27. No Metal Toolchain needed — the prebuilt Inference xcframework bundles the // compiled default.metallib. @@ -31,12 +31,12 @@ struct SDKRelease { // The SDK release this example builds against with no setup. Build against another published // version: set LOCALLM_SDK_VERSION in your shell (works for `swift build` / CI, NOT inside Xcode), // or edit `defaultSDKVersion` here. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ - "2.0.0-dev": SDKRelease( - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/mcp-chat/README.md b/examples/mcp-chat/README.md index 1a33ae9..7afd185 100644 --- a/examples/mcp-chat/README.md +++ b/examples/mcp-chat/README.md @@ -7,7 +7,7 @@ shows Todoist's task list under the call. Qwen3 8B through MLX by default; Apple's on-device model as an alternative (Settings). -`Package.swift` builds against SDK `2.0.0-dev` with no further setup (MCP Apps support is new in +`Package.swift` builds against SDK `2.0.0-GA` with no further setup (MCP Apps support is new in 2.0): `LocalLMLabSDKCore` and `LocalLMLabSDKInference` (the MLX runtime) come from that GitHub Release, `Components` (`../../Components`) and `MCPAppsHost` (`../../MCPAppsHost`) from this repo as source. Needs macOS 27 and Xcode 27 (Swift 6.4). No Metal Toolchain needed — the prebuilt Inference diff --git a/examples/mlx-control-room/Package.swift b/examples/mlx-control-room/Package.swift index e7a5949..cd34f62 100644 --- a/examples/mlx-control-room/Package.swift +++ b/examples/mlx-control-room/Package.swift @@ -25,7 +25,7 @@ struct SDKRelease { // its entry (URL + the `.sha256` next to the zip on the GitHub release) or just replace the strings in // place. Needs a release that includes the model pin / update / cleanup APIs (1.0.0-RC.1 as re-published // on 2026-09-18, or later). -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( @@ -34,11 +34,11 @@ let knownSDKReleases: [String: SDKRelease] = [ inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/model-switch/Package.swift b/examples/model-switch/Package.swift index 09e7f67..ecb21a7 100644 --- a/examples/model-switch/Package.swift +++ b/examples/model-switch/Package.swift @@ -28,16 +28,16 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKRemote-1.0.0-GA.xcframework.zip", remoteChecksum: "294a00eb41a22265849de171930c209d48c27c64c157bc565f76a5e0c13bdb88" ), - "2.0.0-dev": SDKRelease( - remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "a48098d692084ccde5db7c9e2e39806b579b387147a36546995e606baa51099e" + "2.0.0-GA": SDKRelease( + remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", + remoteChecksum: "d54f02c93266d0dd34ea06075af78dd62de353f326ce5e850ff8b8cda5900d8b" ), ] diff --git a/examples/os-matrix/Package.swift b/examples/os-matrix/Package.swift index 1aa39e2..91a331b 100644 --- a/examples/os-matrix/Package.swift +++ b/examples/os-matrix/Package.swift @@ -22,7 +22,7 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( @@ -31,11 +31,11 @@ let knownSDKReleases: [String: SDKRelease] = [ inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/plate-today-tools/Package.swift b/examples/plate-today-tools/Package.swift index 3ff6fdf..603f375 100644 --- a/examples/plate-today-tools/Package.swift +++ b/examples/plate-today-tools/Package.swift @@ -23,16 +23,16 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), - "2.0.0-dev": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" + "2.0.0-GA": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" ), ] diff --git a/examples/plate-today/Package.swift b/examples/plate-today/Package.swift index 1345885..e2a9433 100644 --- a/examples/plate-today/Package.swift +++ b/examples/plate-today/Package.swift @@ -20,16 +20,16 @@ struct SDKRelease { // edit `defaultSDKVersion` here. For a release not listed, add its entry — the URL follows the // pattern below and the checksum is the `.sha256` file next to the zip on that GitHub release — // or just replace the two strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), - "2.0.0-dev": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" + "2.0.0-GA": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" ), ] diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index b3b0fb5..1474683 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -26,7 +26,7 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( @@ -35,11 +35,11 @@ let knownSDKReleases: [String: SDKRelease] = [ inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/repo-qa/Package.swift b/examples/repo-qa/Package.swift index a571878..cf7954c 100644 --- a/examples/repo-qa/Package.swift +++ b/examples/repo-qa/Package.swift @@ -23,16 +23,16 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), - "2.0.0-dev": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" + "2.0.0-GA": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" ), ] diff --git a/examples/security-demo/Package.swift b/examples/security-demo/Package.swift index a890738..727e336 100644 --- a/examples/security-demo/Package.swift +++ b/examples/security-demo/Package.swift @@ -29,16 +29,16 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKRemote-1.0.0-GA.xcframework.zip", remoteChecksum: "294a00eb41a22265849de171930c209d48c27c64c157bc565f76a5e0c13bdb88" ), - "2.0.0-dev": SDKRelease( - remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKRemote-2.0.0-dev.xcframework.zip", - remoteChecksum: "a48098d692084ccde5db7c9e2e39806b579b387147a36546995e606baa51099e" + "2.0.0-GA": SDKRelease( + remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", + remoteChecksum: "d54f02c93266d0dd34ea06075af78dd62de353f326ce5e850ff8b8cda5900d8b" ), ] diff --git a/examples/vistanova/Package.swift b/examples/vistanova/Package.swift index d14eea1..2d3e18e 100644 --- a/examples/vistanova/Package.swift +++ b/examples/vistanova/Package.swift @@ -24,7 +24,7 @@ struct SDKRelease { // strings in place. Needs a release that includes `MLXModelProvider(pinnedRevisions:)` and // `cancelDownload(_:)`: the checksums below are the 1.0.0-RC.1 binaries as re-published on // 2026-09-19. An RC.1 pulled earlier fails the checksum check — re-resolve packages. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( @@ -33,11 +33,11 @@ let knownSDKReleases: [String: SDKRelease] = [ inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/workspace-buddy-local/Package.swift b/examples/workspace-buddy-local/Package.swift index 2ab9713..4ad51f9 100644 --- a/examples/workspace-buddy-local/Package.swift +++ b/examples/workspace-buddy-local/Package.swift @@ -28,7 +28,7 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( @@ -37,11 +37,11 @@ let knownSDKReleases: [String: SDKRelease] = [ inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKInference-1.0.0-GA.xcframework.zip", inferenceChecksum: "6fcbdd5b04fff709e720b92e8b7eda638014a4d19c0dd925e018b72cb7109430" ), - "2.0.0-dev": SDKRelease( - coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - coreChecksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb", - inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKInference-2.0.0-dev.xcframework.zip", - inferenceChecksum: "79a393f494a9f865f3c039b1954eedbd7598bf77b3e50a786e83a297f1f6ed9e" + "2.0.0-GA": SDKRelease( + coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", + inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" ), ] diff --git a/examples/workspace-buddy/Package.swift b/examples/workspace-buddy/Package.swift index c69fc10..ff9e238 100644 --- a/examples/workspace-buddy/Package.swift +++ b/examples/workspace-buddy/Package.swift @@ -21,16 +21,16 @@ struct SDKRelease { // `swift build` / CI, NOT inside Xcode), or edit `defaultSDKVersion` here. For a // release not listed, add its entry (URL + the `.sha256` next to the zip on the // GitHub release) or just replace the strings in place. -let defaultSDKVersion = "2.0.0-dev" +let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2e37dfb2f7925f01ed011262ee5a2aebc561a3ae7a93524acc0285b3e" ), - "2.0.0-dev": SDKRelease( - url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-dev/LocalLMLabSDKCore-2.0.0-dev.xcframework.zip", - checksum: "c741066f04adf052c37202b5a857fbae5e68686a82f167b3af9a1fc565bb78eb" + "2.0.0-GA": SDKRelease( + url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", + checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" ), ] From 75cf09ebd00667dfcbf439acd124af1b3c9af70b Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Wed, 7 Oct 2026 10:04:57 -0700 Subject: [PATCH 61/73] Docs: 2.0.0-GA download links and version README's DMG link and toolkit/README's latest download, verify and unzip steps -> v2.0.0-GA assets (were 1.0.0-GA). migrating-to-2.0 section 2: knownSDKReleases entry, tag and LOCALLM_SDK_VERSION are 2.0.0-GA (2.0.0 is not a known version in the manifests). Co-Authored-By: Claude --- README.md | 2 +- docs/migrating-to-2.0.md | 6 +++--- toolkit/README.md | 10 +++++----- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 96d1cde..a69ab35 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ an Apple entitlement that is still pending, so it is currently inert. **Product page:** [thisbrain.ai/locallm](https://thisbrain.ai/locallm) — or get the app directly: -[LocalLM Lab 1.0.0-GA (arm64 DMG)](https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/LocalLM.Lab-1.0.0-GA-arm64.dmg). +[LocalLM Lab 2.0.0-GA (arm64 DMG)](https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLM.Lab-2.0.0-GA-arm64.dmg). ## LocalLM Lab SDK diff --git a/docs/migrating-to-2.0.md b/docs/migrating-to-2.0.md index 553cb0a..94edc39 100644 --- a/docs/migrating-to-2.0.md +++ b/docs/migrating-to-2.0.md @@ -16,11 +16,11 @@ Most apps change little. The usual `makeSession`, `respond`, `LocalLMLab.Configu ## 2. Point your manifest at the new release -Add `2.0.0` to your manifest's `knownSDKReleases` with the checksums from the release's -`.sha256` assets (tag `v2.0.0`), and build with: +Add `2.0.0-GA` to your manifest's `knownSDKReleases` with the checksums from the release's +`.sha256` assets (tag `v2.0.0-GA`), and build with: ```sh -LOCALLM_SDK_VERSION=2.0.0 swift build +LOCALLM_SDK_VERSION=2.0.0-GA swift build ``` ## 3. Tool authorization: one authorizer, decided per server diff --git a/toolkit/README.md b/toolkit/README.md index 894a898..009c88c 100644 --- a/toolkit/README.md +++ b/toolkit/README.md @@ -17,11 +17,11 @@ release assets on **[github.com/ancientcomputing/locallm-releases](https://github.com/ancientcomputing/locallm-releases/releases)**, alongside the LocalLM Lab app DMG. From **`1.0.0-GA`** on, both ship as release assets on this repo instead, alongside the app DMG and the SDK xcframeworks. -Latest (`1.0.0-GA`): +Latest (`2.0.0-GA`): ```bash -curl -LO https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/localai-toolkit-1.0.0-GA-arm64.zip -curl -LO https://github.com/ancientcomputing/locallm/releases/download/v1.0.0-GA/localai-toolkit-1.0.0-GA-arm64.zip.sha256 +curl -LO https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/localai-toolkit-2.0.0-GA-arm64.zip +curl -LO https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/localai-toolkit-2.0.0-GA-arm64.zip.sha256 ``` (For `1.0.0-beta.4` through `1.0.0-RC.1`, replace the repo/tag above with @@ -39,13 +39,13 @@ curl -LO https://raw.githubusercontent.com/ancientcomputing/locallm/1.0.0-beta/t ## Verify ```bash -shasum -a 256 -c localai-toolkit-1.0.0-GA-arm64.zip.sha256 +shasum -a 256 -c localai-toolkit-2.0.0-GA-arm64.zip.sha256 ``` ## Install ```bash -unzip localai-toolkit-1.0.0-GA-arm64.zip +unzip localai-toolkit-2.0.0-GA-arm64.zip ``` This produces `localai-cli`, `localai-playground-run` (macOS 27), From cdfe98c12291fcb7176897041849f6a4442dfc26 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Wed, 7 Oct 2026 10:16:56 -0700 Subject: [PATCH 62/73] Docs: api-surface from the 2.0.0-GA binaries; tested-models GA note api-surface.md synced from locallmlab-sdk, regenerated from the published v2.0.0-GA xcframeworks (surface unchanged since 2026-10-05; only the version line differs). tested-models: the 2026-10-03 decider run was on a 2.0.0-dev pre-release build and has not been re-run on 2.0.0-GA. Co-Authored-By: Claude --- docs/api-surface.md | 2 +- docs/tested-models.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/api-surface.md b/docs/api-surface.md index 4485c99..7389226 100644 --- a/docs/api-surface.md +++ b/docs/api-surface.md @@ -2,7 +2,7 @@ > Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s. > Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check. -> SDK version: `"2.0.0-dev"` (the published `v2.0.0-dev` xcframeworks, republished 2026-10-05) · generated 2026-10-05T13:29Z +> SDK version: `"2.0.0"` (the published `v2.0.0-GA` xcframeworks, 2026-10-07) · generated 2026-10-07T17:07Z ## LocalLMLabSDKCore diff --git a/docs/tested-models.md b/docs/tested-models.md index 48de775..067e335 100644 --- a/docs/tested-models.md +++ b/docs/tested-models.md @@ -34,7 +34,7 @@ built-in `ClockTool` cross-check. ## As a decider (OpenJev, `lab.decide`) -Point-in-time too: **2026-10-03**, SDK `2.0.0-dev`, a 64 GB Apple-silicon Mac, the SDK's own +Point-in-time too: **2026-10-03**, SDK `2.0.0-dev` (a pre-release build; not re-run on `2.0.0-GA`), a 64 GB Apple-silicon Mac, the SDK's own decision-question evaluation and JevDK's customer-support example. A decider answers typed questions from the probabilities of each option's label, with no generation ([`sdk-guide.md` §6c](sdk-guide.md#6c-decision-models-jev--labdecide)), so what matters is From 0464d8f26e41db386f004f8c46a7135f24f64206 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Wed, 7 Oct 2026 17:34:18 -0700 Subject: [PATCH 63/73] CHANGELOG: 2.0.0 heading carries its GA date (2026-10-07) --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c46a420..096be27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,7 +24,7 @@ recompiling. A breaking change waits for 2.0. (Before RC.1, `1.0.0-beta.N` and e beta.4 → RC.1 changes that can break a build are listed under *Changed — breaking (pre-GA)* in the RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) -## 2.0.0 — unreleased +## 2.0.0 — GA — 2026-10-07 **A one-time breaking release.** From `2.0.0`, every 2.x release is source compatible with it (additive changes only, the same policy 1.x followed). Upgrading: [`docs/migrating-to-2.0.md`](docs/migrating-to-2.0.md). From 6886d17668b02db0c504a02424c961c7c731defb Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Thu, 8 Oct 2026 07:14:39 -0700 Subject: [PATCH 64/73] CHANGELOG 2.0.0: Anthropic streaming crash fix; checksums Fixed: a streamed .anthropicMessages turn crashed the process (usage update sent input 0; FoundationModels traps when a streamed turn's input goes down). Checksums: Core, Claude, Inference from the v2.0.0-GA release; Remote pending its re-publish with the fix. Co-Authored-By: Claude --- CHANGELOG.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 096be27..bb561d2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -211,6 +211,11 @@ API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdec - **`contextBudget`** reported the whole session's cumulative usage as the last turn's, so `fractionUsed` grew every turn. It now reports the latest turn's last model call (the conversation as the model last read it). +- **A streamed Anthropic Messages turn crashed the process** (`LocalLMLabSDKRemote`, `.anthropicMessages`). + Anthropic reports input tokens at the start of a reply and output tokens at the end; the second usage + update sent input as 0, and FoundationModels traps when a streamed turn's input count goes down — so + the process stopped right after the reply's text. A non-streamed turn didn't crash but reported 0 input + tokens. Input is now carried forward. The OpenAI dialects weren't affected. Also present in 1.0.0. ### Added — examples @@ -228,7 +233,12 @@ API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdec ### Checksums (SHA-256) -_Filled in at release._ +``` +LocalLMLabSDKCore-2.0.0-GA.xcframework.zip faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029 +LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip 7afe7d79c63b67cc695d53a1378d5eee23fd508d2409908a717085622b86a771 +LocalLMLabSDKInference-2.0.0-GA.xcframework.zip fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8 +LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip (re-published with the Anthropic fix above; filled in when published) +``` --- From 936d2e5a08ccd136b18f105225ce659da16408c6 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Thu, 8 Oct 2026 17:28:40 -0700 Subject: [PATCH 65/73] Cascade re-published 2.0.0-GA SDK checksums All four v2.0.0-GA xcframeworks were re-published on 2026-10-08 (Remote with the Anthropic streaming fix; Core, Claude, Inference rebuilt with it). Each checksum is from the release's .sha256, verified against the downloaded zip: Core 246bad16, Claude e1c50ad8, Inference 60d3f367, Remote 901198d2. 25 pins in Components and the 15 SDK examples; examples/README.md's samples; CHANGELOG 2.0.0 checksum table (Remote filled in) and heading note the re-publish; api-surface version line (surface unchanged). Built: Components, MCPAppsHost and every example with swift build (fresh scratch path, after scripts/flush-sdk-cache.sh), VistaNova with xcodebuild. Co-Authored-By: Claude --- CHANGELOG.md | 10 +++++----- Components/Package.swift | 2 +- docs/api-surface.md | 2 +- examples/README.md | 6 +++--- examples/aiql/Package.swift | 4 ++-- examples/code-buddy/Package.swift | 4 ++-- examples/jevdk/Package.swift | 6 +++--- examples/mcp-chat/Package.swift | 2 +- examples/mlx-control-room/Package.swift | 4 ++-- examples/model-switch/Package.swift | 2 +- examples/os-matrix/Package.swift | 4 ++-- examples/plate-today-tools/Package.swift | 2 +- examples/plate-today/Package.swift | 2 +- examples/repo-qa-local/Package.swift | 4 ++-- examples/repo-qa/Package.swift | 2 +- examples/security-demo/Package.swift | 2 +- examples/vistanova/Package.swift | 4 ++-- examples/workspace-buddy-local/Package.swift | 4 ++-- examples/workspace-buddy/Package.swift | 2 +- 19 files changed, 34 insertions(+), 34 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bb561d2..6071290 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,7 +24,7 @@ recompiling. A breaking change waits for 2.0. (Before RC.1, `1.0.0-beta.N` and e beta.4 → RC.1 changes that can break a build are listed under *Changed — breaking (pre-GA)* in the RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) -## 2.0.0 — GA — 2026-10-07 +## 2.0.0 — GA — 2026-10-07 (binaries re-published 2026-10-08) **A one-time breaking release.** From `2.0.0`, every 2.x release is source compatible with it (additive changes only, the same policy 1.x followed). Upgrading: [`docs/migrating-to-2.0.md`](docs/migrating-to-2.0.md). @@ -234,10 +234,10 @@ API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdec ### Checksums (SHA-256) ``` -LocalLMLabSDKCore-2.0.0-GA.xcframework.zip faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029 -LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip 7afe7d79c63b67cc695d53a1378d5eee23fd508d2409908a717085622b86a771 -LocalLMLabSDKInference-2.0.0-GA.xcframework.zip fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8 -LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip (re-published with the Anthropic fix above; filled in when published) +LocalLMLabSDKCore-2.0.0-GA.xcframework.zip 246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1 +LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip e1c50ad8f14550937d57766fcf3d13008f2e5487d64b0b0e52e9ae8b47d63562 +LocalLMLabSDKInference-2.0.0-GA.xcframework.zip 60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c +LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip 901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489 ``` --- diff --git a/Components/Package.swift b/Components/Package.swift index 0099868..7a5e3b5 100644 --- a/Components/Package.swift +++ b/Components/Package.swift @@ -33,7 +33,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" + checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" ), ] diff --git a/docs/api-surface.md b/docs/api-surface.md index 7389226..28bb8ae 100644 --- a/docs/api-surface.md +++ b/docs/api-surface.md @@ -2,7 +2,7 @@ > Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s. > Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check. -> SDK version: `"2.0.0"` (the published `v2.0.0-GA` xcframeworks, 2026-10-07) · generated 2026-10-07T17:07Z +> SDK version: `"2.0.0"` (the published `v2.0.0-GA` xcframeworks, re-published 2026-10-08) · generated 2026-10-09T00:26Z ## LocalLMLabSDKCore diff --git a/examples/README.md b/examples/README.md index aa8e31d..1c8cc46 100644 --- a/examples/README.md +++ b/examples/README.md @@ -177,7 +177,7 @@ let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease(url: "…/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2…"), "2.0.0-GA": SDKRelease(url: "…/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "faa92a02…"), + checksum: "246bad16…"), ] ``` @@ -189,9 +189,9 @@ distinct `.xcframework.zip.sha256`; don't reuse Core's for another module): ```bash curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip.sha256 -# → faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029 +# → 246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1 curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip.sha256 -# → 7afe7d79c63b67cc695d53a1378d5eee23fd508d2409908a717085622b86a771 +# → e1c50ad8f14550937d57766fcf3d13008f2e5487d64b0b0e52e9ae8b47d63562 ``` Then point `defaultSDKVersion` at it (works everywhere, Xcode included) or pass diff --git a/examples/aiql/Package.swift b/examples/aiql/Package.swift index b339922..b4fad1f 100644 --- a/examples/aiql/Package.swift +++ b/examples/aiql/Package.swift @@ -42,9 +42,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/code-buddy/Package.swift b/examples/code-buddy/Package.swift index 35cfd43..cf6d3f0 100644 --- a/examples/code-buddy/Package.swift +++ b/examples/code-buddy/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/jevdk/Package.swift b/examples/jevdk/Package.swift index 4c4b388..f47d554 100644 --- a/examples/jevdk/Package.swift +++ b/examples/jevdk/Package.swift @@ -38,11 +38,11 @@ let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", - remoteChecksum: "d54f02c93266d0dd34ea06075af78dd62de353f326ce5e850ff8b8cda5900d8b", + remoteChecksum: "901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift index 8987e82..db4b92a 100644 --- a/examples/mcp-chat/Package.swift +++ b/examples/mcp-chat/Package.swift @@ -36,7 +36,7 @@ let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-GA": SDKRelease( inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/mlx-control-room/Package.swift b/examples/mlx-control-room/Package.swift index cd34f62..cf098b7 100644 --- a/examples/mlx-control-room/Package.swift +++ b/examples/mlx-control-room/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/model-switch/Package.swift b/examples/model-switch/Package.swift index ecb21a7..df11c08 100644 --- a/examples/model-switch/Package.swift +++ b/examples/model-switch/Package.swift @@ -37,7 +37,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", - remoteChecksum: "d54f02c93266d0dd34ea06075af78dd62de353f326ce5e850ff8b8cda5900d8b" + remoteChecksum: "901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489" ), ] diff --git a/examples/os-matrix/Package.swift b/examples/os-matrix/Package.swift index 91a331b..2c0c5be 100644 --- a/examples/os-matrix/Package.swift +++ b/examples/os-matrix/Package.swift @@ -33,9 +33,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/plate-today-tools/Package.swift b/examples/plate-today-tools/Package.swift index 603f375..fbd36b2 100644 --- a/examples/plate-today-tools/Package.swift +++ b/examples/plate-today-tools/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" + checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" ), ] diff --git a/examples/plate-today/Package.swift b/examples/plate-today/Package.swift index e2a9433..f620c60 100644 --- a/examples/plate-today/Package.swift +++ b/examples/plate-today/Package.swift @@ -29,7 +29,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" + checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" ), ] diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index 1474683..aa956d3 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -37,9 +37,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/repo-qa/Package.swift b/examples/repo-qa/Package.swift index cf7954c..1d8ea51 100644 --- a/examples/repo-qa/Package.swift +++ b/examples/repo-qa/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" + checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" ), ] diff --git a/examples/security-demo/Package.swift b/examples/security-demo/Package.swift index 727e336..b25672b 100644 --- a/examples/security-demo/Package.swift +++ b/examples/security-demo/Package.swift @@ -38,7 +38,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", - remoteChecksum: "d54f02c93266d0dd34ea06075af78dd62de353f326ce5e850ff8b8cda5900d8b" + remoteChecksum: "901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489" ), ] diff --git a/examples/vistanova/Package.swift b/examples/vistanova/Package.swift index 2d3e18e..eeb26a9 100644 --- a/examples/vistanova/Package.swift +++ b/examples/vistanova/Package.swift @@ -35,9 +35,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/workspace-buddy-local/Package.swift b/examples/workspace-buddy-local/Package.swift index 4ad51f9..b9ce009 100644 --- a/examples/workspace-buddy-local/Package.swift +++ b/examples/workspace-buddy-local/Package.swift @@ -39,9 +39,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029", + coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "fde8babbb0a8512b3d26e3d3bcc8139d02eb00375b2fbd6ed1e9157dfec55db8" + inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" ), ] diff --git a/examples/workspace-buddy/Package.swift b/examples/workspace-buddy/Package.swift index ff9e238..2bee0ff 100644 --- a/examples/workspace-buddy/Package.swift +++ b/examples/workspace-buddy/Package.swift @@ -30,7 +30,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "faa92a02bc7e5b0c2de284507e730798042bc7154a51d361535a7ba801d54029" + checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" ), ] From 3dc067c9440ac9156ac79a17ab4a932b6b08d79c Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 9 Oct 2026 07:27:11 -0700 Subject: [PATCH 66/73] CHANGELOG 2.0.0: loadTable recordsAt fallback Co-Authored-By: Claude --- CHANGELOG.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6071290..dcf77be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -211,6 +211,13 @@ API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdec - **`contextBudget`** reported the whole session's cumulative usage as the last turn's, so `fractionUsed` grew every turn. It now reports the latest turn's last model call (the conversation as the model last read it). +- **`loadTable` let a model guess `recordsAt` until the tool-loop limit stopped it** (`LocalLMLabSDKCore`). + A `recordsAt` that isn't an array in the JSON was an error that didn't say which path would work, so a + small model guessed again (`"data"`, `"data.items"`, …). 2.0's 16-call limit on a local model's turn + then stopped about half the runs of the [`aiql`](examples/aiql/) walkthrough; without the limit (1.0) + they succeeded after 10–42 calls. `loadTable` now falls back to auto-detection when `recordsAt` misses + and says so in its reply; when that's ambiguous the error lists the candidate arrays. The walkthrough + now takes three model calls. - **A streamed Anthropic Messages turn crashed the process** (`LocalLMLabSDKRemote`, `.anthropicMessages`). Anthropic reports input tokens at the start of a reply and output tokens at the end; the second usage update sent input as 0, and FoundationModels traps when a streamed turn's input count goes down — so From 4500dd16df3bd17309899991011891efa7f10411 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 9 Oct 2026 07:39:21 -0700 Subject: [PATCH 67/73] aiql README: Qwen3.8-27B-4bit as the option for a larger Mac Measured with a headless copy of the pipeline (fixed loadTable): 3/3, 4 calls, ~37 s, following every step and keeping every requested column; the 14B default 10/10, 3 calls, 14 s. 15 GB of weights clears the 70% preflight on a 24 GB Mac. Co-Authored-By: Claude --- examples/aiql/README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/examples/aiql/README.md b/examples/aiql/README.md index 5786287..764bf30 100644 --- a/examples/aiql/README.md +++ b/examples/aiql/README.md @@ -278,6 +278,12 @@ size-vs-memory preflight (weights ≤ 70% of physical RAM). ranges, a child-table join, a two-dataset `JOIN`, `GROUP BY … HAVING`. A bigger model mainly buys a touch more reliability on request→column matching. A 4B model usually works. +`mlx-community/Qwen3.8-27B-4bit` (~15 GB) follows the pipeline more closely: in our runs it left +`recordsAt` empty as told, ran every step, and kept every column the request named (the usage +index as well as the automation percentage). It's about 2.5× slower than the 14B (about 37 s +against 14 s for the walkthrough on an M5 Max) and needs a 24 GB Mac to clear the preflight — +32 GB to be comfortable. Type it in the Model field. + The session is built with `options: SessionOptions(effort: .off)` — the Qwen3 chat template's thinking toggle is turned off, so the model skips its `…` pass and answers faster. The pipeline is mechanical enough (pick a tool, name a table, write one `SELECT`) that From a149e3c27681b147d245ff333a8069e8410da957 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 9 Oct 2026 07:48:59 -0700 Subject: [PATCH 68/73] Cascade 2.0.0-GA SDK checksums (re-published 2026-10-09, loadTable fix) All four v2.0.0-GA xcframeworks re-published on 2026-10-09; Core carries the loadTable recordsAt fallback. Checksums from the release's .sha256, verified against the downloaded zips: Core 5f9418a6, Claude f235480f, Inference d2ed4a02, Remote 1e809cff. 25 pins in Components and the 15 SDK examples; examples/README.md samples; CHANGELOG checksum table and heading; api-surface version line (surface unchanged). Built: Components, MCPAppsHost and every example with swift build (fresh scratch path, after scripts/flush-sdk-cache.sh), VistaNova with xcodebuild. AIQL's walkthrough on the published Core + Inference (headless copy of its pipeline, Qwen3-14B-4bit): 3/3, 3 model calls, 14 s. Co-Authored-By: Claude --- CHANGELOG.md | 10 +++++----- Components/Package.swift | 2 +- docs/api-surface.md | 2 +- examples/README.md | 6 +++--- examples/aiql/Package.swift | 4 ++-- examples/code-buddy/Package.swift | 4 ++-- examples/jevdk/Package.swift | 6 +++--- examples/mcp-chat/Package.swift | 2 +- examples/mlx-control-room/Package.swift | 4 ++-- examples/model-switch/Package.swift | 2 +- examples/os-matrix/Package.swift | 4 ++-- examples/plate-today-tools/Package.swift | 2 +- examples/plate-today/Package.swift | 2 +- examples/repo-qa-local/Package.swift | 4 ++-- examples/repo-qa/Package.swift | 2 +- examples/security-demo/Package.swift | 2 +- examples/vistanova/Package.swift | 4 ++-- examples/workspace-buddy-local/Package.swift | 4 ++-- examples/workspace-buddy/Package.swift | 2 +- 19 files changed, 34 insertions(+), 34 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dcf77be..7bb8e27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,7 +24,7 @@ recompiling. A breaking change waits for 2.0. (Before RC.1, `1.0.0-beta.N` and e beta.4 → RC.1 changes that can break a build are listed under *Changed — breaking (pre-GA)* in the RC.1 entry, and each entry's own "Beta caveats" apply to the betas.) -## 2.0.0 — GA — 2026-10-07 (binaries re-published 2026-10-08) +## 2.0.0 — GA — 2026-10-07 (binaries re-published 2026-10-08 and 2026-10-09) **A one-time breaking release.** From `2.0.0`, every 2.x release is source compatible with it (additive changes only, the same policy 1.x followed). Upgrading: [`docs/migrating-to-2.0.md`](docs/migrating-to-2.0.md). @@ -241,10 +241,10 @@ API: [`docs/sdk-guide.md` §6c](docs/sdk-guide.md#6c-decision-models-jev--labdec ### Checksums (SHA-256) ``` -LocalLMLabSDKCore-2.0.0-GA.xcframework.zip 246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1 -LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip e1c50ad8f14550937d57766fcf3d13008f2e5487d64b0b0e52e9ae8b47d63562 -LocalLMLabSDKInference-2.0.0-GA.xcframework.zip 60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c -LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip 901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489 +LocalLMLabSDKCore-2.0.0-GA.xcframework.zip 5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b +LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip f235480fd4842e91c0c17b416eb4048b7abe6b3c2d41f332984c5e7af368092e +LocalLMLabSDKInference-2.0.0-GA.xcframework.zip d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185 +LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip 1e809cff19a7d232cd82fd36eb3ba7d79a93b55f71b6694041c9d9e3f6261be9 ``` --- diff --git a/Components/Package.swift b/Components/Package.swift index 7a5e3b5..3c55ec4 100644 --- a/Components/Package.swift +++ b/Components/Package.swift @@ -33,7 +33,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" + checksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b" ), ] diff --git a/docs/api-surface.md b/docs/api-surface.md index 28bb8ae..cf9bbb4 100644 --- a/docs/api-surface.md +++ b/docs/api-surface.md @@ -2,7 +2,7 @@ > Machine-generated by `scripts/dump-public-api.sh` from the built `.swiftinterface`s. > Do not hand-edit. `sdk-guide.md` §12 is the *narrative* reference; this is the check. -> SDK version: `"2.0.0"` (the published `v2.0.0-GA` xcframeworks, re-published 2026-10-08) · generated 2026-10-09T00:26Z +> SDK version: `"2.0.0"` (the published `v2.0.0-GA` xcframeworks, re-published 2026-10-09) · generated 2026-10-09T14:46Z ## LocalLMLabSDKCore diff --git a/examples/README.md b/examples/README.md index 1c8cc46..fa7c80f 100644 --- a/examples/README.md +++ b/examples/README.md @@ -177,7 +177,7 @@ let knownSDKReleases: [String: SDKRelease] = [ "1.0.0-GA": SDKRelease(url: "…/v1.0.0-GA/LocalLMLabSDKCore-1.0.0-GA.xcframework.zip", checksum: "7d77a9c2…"), "2.0.0-GA": SDKRelease(url: "…/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "246bad16…"), + checksum: "5f9418a6…"), ] ``` @@ -189,9 +189,9 @@ distinct `.xcframework.zip.sha256`; don't reuse Core's for another module): ```bash curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip.sha256 -# → 246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1 +# → 5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b curl -sL https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKClaude-2.0.0-GA.xcframework.zip.sha256 -# → e1c50ad8f14550937d57766fcf3d13008f2e5487d64b0b0e52e9ae8b47d63562 +# → f235480fd4842e91c0c17b416eb4048b7abe6b3c2d41f332984c5e7af368092e ``` Then point `defaultSDKVersion` at it (works everywhere, Xcode included) or pass diff --git a/examples/aiql/Package.swift b/examples/aiql/Package.swift index b4fad1f..2940a5b 100644 --- a/examples/aiql/Package.swift +++ b/examples/aiql/Package.swift @@ -42,9 +42,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/code-buddy/Package.swift b/examples/code-buddy/Package.swift index cf6d3f0..b0eb6e3 100644 --- a/examples/code-buddy/Package.swift +++ b/examples/code-buddy/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/jevdk/Package.swift b/examples/jevdk/Package.swift index f47d554..a0db85a 100644 --- a/examples/jevdk/Package.swift +++ b/examples/jevdk/Package.swift @@ -38,11 +38,11 @@ let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", - remoteChecksum: "901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489", + remoteChecksum: "1e809cff19a7d232cd82fd36eb3ba7d79a93b55f71b6694041c9d9e3f6261be9", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/mcp-chat/Package.swift b/examples/mcp-chat/Package.swift index db4b92a..51c02fc 100644 --- a/examples/mcp-chat/Package.swift +++ b/examples/mcp-chat/Package.swift @@ -36,7 +36,7 @@ let defaultSDKVersion = "2.0.0-GA" let knownSDKReleases: [String: SDKRelease] = [ "2.0.0-GA": SDKRelease( inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/mlx-control-room/Package.swift b/examples/mlx-control-room/Package.swift index cf098b7..8149257 100644 --- a/examples/mlx-control-room/Package.swift +++ b/examples/mlx-control-room/Package.swift @@ -36,9 +36,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/model-switch/Package.swift b/examples/model-switch/Package.swift index df11c08..6badf07 100644 --- a/examples/model-switch/Package.swift +++ b/examples/model-switch/Package.swift @@ -37,7 +37,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", - remoteChecksum: "901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489" + remoteChecksum: "1e809cff19a7d232cd82fd36eb3ba7d79a93b55f71b6694041c9d9e3f6261be9" ), ] diff --git a/examples/os-matrix/Package.swift b/examples/os-matrix/Package.swift index 2c0c5be..2d2ff5e 100644 --- a/examples/os-matrix/Package.swift +++ b/examples/os-matrix/Package.swift @@ -33,9 +33,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/plate-today-tools/Package.swift b/examples/plate-today-tools/Package.swift index fbd36b2..2e034e0 100644 --- a/examples/plate-today-tools/Package.swift +++ b/examples/plate-today-tools/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" + checksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b" ), ] diff --git a/examples/plate-today/Package.swift b/examples/plate-today/Package.swift index f620c60..aac2cdd 100644 --- a/examples/plate-today/Package.swift +++ b/examples/plate-today/Package.swift @@ -29,7 +29,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" + checksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b" ), ] diff --git a/examples/repo-qa-local/Package.swift b/examples/repo-qa-local/Package.swift index aa956d3..0e159ff 100644 --- a/examples/repo-qa-local/Package.swift +++ b/examples/repo-qa-local/Package.swift @@ -37,9 +37,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/repo-qa/Package.swift b/examples/repo-qa/Package.swift index 1d8ea51..770add5 100644 --- a/examples/repo-qa/Package.swift +++ b/examples/repo-qa/Package.swift @@ -32,7 +32,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" + checksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b" ), ] diff --git a/examples/security-demo/Package.swift b/examples/security-demo/Package.swift index b25672b..4fabfdc 100644 --- a/examples/security-demo/Package.swift +++ b/examples/security-demo/Package.swift @@ -38,7 +38,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( remoteURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKRemote-2.0.0-GA.xcframework.zip", - remoteChecksum: "901198d2b1080bbb8ef13bdb7aecca5f9374f88af8c5361ad9e61e8052d58489" + remoteChecksum: "1e809cff19a7d232cd82fd36eb3ba7d79a93b55f71b6694041c9d9e3f6261be9" ), ] diff --git a/examples/vistanova/Package.swift b/examples/vistanova/Package.swift index eeb26a9..ce338d3 100644 --- a/examples/vistanova/Package.swift +++ b/examples/vistanova/Package.swift @@ -35,9 +35,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/workspace-buddy-local/Package.swift b/examples/workspace-buddy-local/Package.swift index b9ce009..a8b092e 100644 --- a/examples/workspace-buddy-local/Package.swift +++ b/examples/workspace-buddy-local/Package.swift @@ -39,9 +39,9 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( coreURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - coreChecksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1", + coreChecksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b", inferenceURL: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKInference-2.0.0-GA.xcframework.zip", - inferenceChecksum: "60d3f367338adb8093b705b776b68f2ca27fdec75f128b304d280c78f48c069c" + inferenceChecksum: "d2ed4a02686aceab0edf4231ede55c924ea3ab273c7c5cab0bb02f4bda40a185" ), ] diff --git a/examples/workspace-buddy/Package.swift b/examples/workspace-buddy/Package.swift index 2bee0ff..235c799 100644 --- a/examples/workspace-buddy/Package.swift +++ b/examples/workspace-buddy/Package.swift @@ -30,7 +30,7 @@ let knownSDKReleases: [String: SDKRelease] = [ ), "2.0.0-GA": SDKRelease( url: "https://github.com/ancientcomputing/locallm/releases/download/v2.0.0-GA/LocalLMLabSDKCore-2.0.0-GA.xcframework.zip", - checksum: "246bad16b8a39dc72e037778d7e7128fe2742ea7cb7b19dd2d456b94bae131b1" + checksum: "5f9418a6879b22227afdf867cf474d426b93ae36d783d5a958efaae1e347681b" ), ] From 2a6d9cd0fc029bd7661e28edcb602dd60aa42e17 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 9 Oct 2026 08:05:48 -0700 Subject: [PATCH 69/73] components-updates-demo: SDK 2.0.0-GA in the README; come to the front on swift run README's "Getting the SDK" said the example tracks and builds against 1.0.0-RC.1; it builds against Components' default, 2.0.0-GA. Launched with swift run there's no app bundle, so the window could open behind the terminal; the app now claims regular-app status and activates at launch (same as mlx-control-room), and the README says so instead of "click it". Co-Authored-By: Claude --- examples/components-updates-demo/README.md | 8 ++++---- .../Sources/UpdatesDemo/UpdatesDemoApp.swift | 12 ++++++++++++ 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/examples/components-updates-demo/README.md b/examples/components-updates-demo/README.md index 38c053c..b44035a 100644 --- a/examples/components-updates-demo/README.md +++ b/examples/components-updates-demo/README.md @@ -25,9 +25,9 @@ Requires macOS 27+ on Apple Silicon (Xcode 27 to build). No permissions, no sign ## Getting the SDK -This branch tracks `1.0.0-RC.1`. Nothing to download by hand — `Package.swift` depends on the sibling +This branch tracks `2.0.0-GA`. Nothing to download by hand — `Package.swift` depends on the sibling [`Components`](../../Components/) package, which resolves `LocalLMLabSDKCore` as a binary dependency and builds -against `1.0.0-RC.1` by default. Set `LOCALLM_SDK_VERSION` in a shell (not Xcode) to pin another published +against `2.0.0-GA` by default. Set `LOCALLM_SDK_VERSION` in a shell (not Xcode) to pin another published release — see [`../README.md`](../README.md#building--running-an-sdk-example). ## Build and run @@ -38,8 +38,8 @@ DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer swift run UpdatesDemo Skip `DEVELOPER_DIR` if `xcode-select -p` already points at Xcode 27 (adjust the path if yours lives elsewhere). There is no `.xcodeproj` and no `packaging/`; to run it from Xcode, open `Package.swift` and Run the -**UpdatesDemo** scheme. Started with `swift run`, the window is not part of an app bundle, so if it doesn't -come to the front, click it. +**UpdatesDemo** scheme. Started with `swift run` there's no app bundle; the app makes itself a regular app +and comes to the front at launch. ## What to try diff --git a/examples/components-updates-demo/Sources/UpdatesDemo/UpdatesDemoApp.swift b/examples/components-updates-demo/Sources/UpdatesDemo/UpdatesDemoApp.swift index 6344e44..15cde18 100644 --- a/examples/components-updates-demo/Sources/UpdatesDemo/UpdatesDemoApp.swift +++ b/examples/components-updates-demo/Sources/UpdatesDemo/UpdatesDemoApp.swift @@ -1,3 +1,4 @@ +import AppKit import LocalLMLabSDKComponents import LocalLMLabSDKCore import SwiftUI @@ -10,6 +11,17 @@ private let newerCommit = "ff1143e3a10547c9f2129e94ca37059b096b23f4" @main @available(macOS 27, *) struct UpdatesDemoApp: App { + init() { + // Launched with `swift run` there is no app bundle, so macOS starts this as a background-style + // process whose window can open behind the terminal. Claim regular-app status and come to the + // front — only when it isn't one already (re-setting it logs "Task policy set failed"). Same as + // mlx-control-room. + if NSApplication.shared.activationPolicy() != .regular { + NSApplication.shared.setActivationPolicy(.regular) + } + DispatchQueue.main.async { NSApplication.shared.activate() } + } + var body: some Scene { WindowGroup("Components: onboarding, updates, versions") { DemoView().frame(minWidth: 640, minHeight: 720) From f7597a5e6be9771d9b22d76f6e675c446a319754 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Fri, 9 Oct 2026 09:02:24 -0700 Subject: [PATCH 70/73] model-switch README: how to write each provider's model id The Add model id box wants the provider's API id, not its display name; OpenRouter's is vendor/model (openai/gpt-6.1-sol, not "GPT-6.1 Sol" or gpt-6.1-sol) and is copied from the model's page on openrouter.ai. A table of format, example and where to find it per provider; Test connection checks each id. Co-Authored-By: Claude --- examples/model-switch/README.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/examples/model-switch/README.md b/examples/model-switch/README.md index bc132db..a7c089c 100644 --- a/examples/model-switch/README.md +++ b/examples/model-switch/README.md @@ -35,6 +35,20 @@ Running a packaged build (`packaging/build-and-sign.sh`, below): 4. **Switch models mid-conversation** — the picker changes which provider the *next* turn routes to; the transcript carries over. +**Model ids.** The *Add model id* box wants the provider's API id, not the name it shows on its +website. Each provider writes it differently: + +| Provider | Format | Example | Where to find it | +|---|---|---|---| +| OpenRouter | `vendor/model`, lowercase, hyphens | `openai/gpt-6.1-sol`, `anthropic/claude-sonnet-5` | the model's page on openrouter.ai — the id under its name, with a copy button | +| OpenAI | the bare model id | `gpt-6-astra` | OpenAI's models page in its API docs | +| Anthropic | the bare model id | `claude-sonnet-5` | Anthropic's models overview in its API docs | +| Custom server | whatever it reports | — | `GET /v1/models` on the server | + +So for OpenRouter's "OpenAI: GPT-6.1 Sol", type `openai/gpt-6.1-sol` — not `GPT-6.1 Sol`, not +`gpt-6.1 sol`, and not `gpt-6.1-sol` without the vendor. **Test connection** checks each id +against the provider's list and says which one it doesn't know. + **Prompts to try.** With only **system** configured (no key), any plain chat prompt works — `Explain the difference between a struct and a class in Swift` — but there's nothing to switch between and web search does nothing. The example only gets interesting once a provider with an From 2bf2e9df1384b4f0a13b6b563b1a41979c266cfa Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sat, 10 Oct 2026 06:37:55 -0700 Subject: [PATCH 71/73] sdk-guide: the workspace tools are Core's, and the app owns the folder access MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The WorkspaceAccess passage called the tools "FoundationModels Tools", which read as if Apple ships them; they're Core's, conforming to FoundationModels' Tool protocol. It also said WorkspaceAccess owns the security-scoped-bookmark bracket; the app does (picker, bookmark, startAccessingSecurityScopedResource — §8), and WorkspaceAccess keeps each read and write inside that folder. Examples line: workspace-buddy-local added, and which tools each example offers. Co-Authored-By: Claude --- docs/sdk-guide.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/sdk-guide.md b/docs/sdk-guide.md index 5aa94c5..13f2cab 100644 --- a/docs/sdk-guide.md +++ b/docs/sdk-guide.md @@ -1737,14 +1737,18 @@ side-by-side output. ### `WorkspaceAccess` + the Workspace tools — let the model touch files > **Use it when** the model needs to read or edit files in a folder the user picked. -> `WorkspaceAccess` owns the security-scoped-bookmark bracket; the ready-made tools -> (`SearchWorkspaceTool`, `WorkspaceTreeTool`, `ReadWorkspaceFileTool`, `ReadFileRangeTool`, -> `ListWorkspaceFilesTool`, `ApplyPatchTool`, `EditWorkspaceFileTool`, `WriteWorkspaceFileTool`, -> `DeleteWorkspaceFileTool`) are FoundationModels `Tool`s you drop straight into `makeSession`. +> Your app picks the folder and holds its security-scoped access open +> ([§8](#8-filesystem-access-security-scoped-bookmarks-example-not-in-core)); `WorkspaceAccess` +> keeps every read and write inside that folder. The ready-made tools (`SearchWorkspaceTool`, +> `WorkspaceTreeTool`, `ReadWorkspaceFileTool`, `ReadFileRangeTool`, `ListWorkspaceFilesTool`, +> `ApplyPatchTool`, `EditWorkspaceFileTool`, `WriteWorkspaceFileTool`, `DeleteWorkspaceFileTool`) +> are part of Core, not Apple's: they conform to FoundationModels' `Tool` protocol, so you pass +> them straight into `makeSession`. > -> **Examples that use it:** [`code-buddy`](../examples/code-buddy/) and -> [`workspace-buddy`](../examples/workspace-buddy/) (the Core-only, no-MLX version) both use -> these. +> **Examples that use it:** [`code-buddy`](../examples/code-buddy/) offers seven of them (all but +> `ApplyPatchTool` and `DeleteWorkspaceFileTool`); [`workspace-buddy`](../examples/workspace-buddy/) +> (Core only, no MLX) and [`workspace-buddy-local`](../examples/workspace-buddy-local/) (a local MLX +> model) offer four: list, read, write and edit. ## 6b. Online providers — GPT, Claude online, OpenRouter (`LocalLMLabSDKRemote`) From 886939cc5fd802f97b69243ef47063c351d81dc0 Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Sun, 11 Oct 2026 11:45:53 -0700 Subject: [PATCH 72/73] localai-cli READMEs: where the model comes from; current download location first Python and Swift: localai-cli always uses the AI Models choice, from LocalLM Lab's own app-config.json, even with a different --config file, and ignores a request's "model" (verified: "model": "system" still answered on the app's MLX model); every response's "model" says which one answered. Setup step 2 leads with this repo's releases (1.0.0-GA on); locallm-releases and toolkit/ are listed as the places for older versions. Co-Authored-By: Claude --- examples/localai-cli-swift/README.md | 21 +++++++++++---------- examples/localai-cli/README.md | 22 ++++++++++++---------- 2 files changed, 23 insertions(+), 20 deletions(-) diff --git a/examples/localai-cli-swift/README.md b/examples/localai-cli-swift/README.md index 0b3e0d0..7c87d55 100644 --- a/examples/localai-cli-swift/README.md +++ b/examples/localai-cli-swift/README.md @@ -41,9 +41,11 @@ the user picks which in the **AI Models** screen: On **macOS 26** only **Apple on-device** (`system`) is available — Private Cloud Compute, Claude, and open-weight (MLX) models need macOS 27. -`localai-cli` reads that choice from `app-config.json` — there is no -`--model` flag. To change which model your scripts use, change the default -in **AI Models**. +`localai-cli` always uses the model chosen in **AI Models** — the `default_model` in +LocalLM Lab's own `~/Library/Application Support/LocalLM Lab/app-config.json`, even when +you pass a different file with `--config`. There is no `--model` flag, and a `"model"` +field in the request is ignored. To change which model your scripts use, change the +default in **AI Models**. Every response's `"model"` field says which model answered. ### How connector / MCP access works @@ -152,13 +154,12 @@ file is this repo's script-runner boilerplate, not something to copy. `~/Library/Application Support/LocalLM Lab/app-config.json` — `localai-cli` only reads it. Connector and MCP calls run inside LocalLM Lab's own process, so the app must be **running**, not just installed. -2. Get the `localai-toolkit--arm64.zip` — a release asset on - [`ancientcomputing/locallm-releases`](https://github.com/ancientcomputing/locallm-releases/releases) - for `1.0.0-beta.4` through `1.0.0-RC.1`, on this repo's own - [releases](https://github.com/ancientcomputing/locallm/releases) from - `1.0.0-GA` on, or [`../../toolkit/`](../../toolkit/) for older - versions (see that folder's README) — and unzip it. `localai-cli` and - `localai-playground-run` are a matched pair. +2. Get the `localai-toolkit--arm64.zip` from this repo's + [releases](https://github.com/ancientcomputing/locallm/releases) (`1.0.0-GA` on) and + unzip it. `localai-cli` and `localai-playground-run` are a matched pair. Older + versions: `1.0.0-beta.4` through `1.0.0-RC.1` are on + [`ancientcomputing/locallm-releases`](https://github.com/ancientcomputing/locallm-releases/releases), + earlier ones in [`../../toolkit/`](../../toolkit/) (see that folder's README). 3. Put both binaries in this folder (or set `LOCALAI_CLI_PATH`). ## Run diff --git a/examples/localai-cli/README.md b/examples/localai-cli/README.md index 4a81c58..cdf17ce 100644 --- a/examples/localai-cli/README.md +++ b/examples/localai-cli/README.md @@ -32,9 +32,12 @@ the user picks which in the **AI Models** screen: On **macOS 26** only **Apple on-device** (`system`) is available — Private Cloud Compute, Claude, and open-weight (MLX) models need macOS 27. -`localai-cli` reads that choice from the config file (below) — there is no -`--model` flag. To change which model your scripts use, change the default -in **AI Models**; the next `localai-cli` call picks it up. +`localai-cli` always uses the model chosen in **AI Models** — the `default_model` in +LocalLM Lab's own `~/Library/Application Support/LocalLM Lab/app-config.json`, even when +you pass a different file with `--config`. There is no `--model` flag, and a `"model"` +field in the request is ignored. To change which model your scripts use, change the +default in **AI Models**; the next `localai-cli` call picks it up. Every response's +`"model"` field says which model answered. ### How connector / MCP access works @@ -64,13 +67,12 @@ for that one call, via `connectors` and `mcp_tools` fields: `localai-cli` only ever reads this file, never creates or edits it. Connector and MCP calls also run inside LocalLM Lab's own process (reached over a local socket), so the app must be **running**, not just installed. -2. Get the `localai-toolkit--arm64.zip` — a release asset on - [`ancientcomputing/locallm-releases`](https://github.com/ancientcomputing/locallm-releases/releases) - for `1.0.0-beta.4` through `1.0.0-RC.1`, on this repo's own - [releases](https://github.com/ancientcomputing/locallm/releases) from - `1.0.0-GA` on, or [`../../toolkit/`](../../toolkit/) for older - versions (see that folder's README) — and unzip it. `localai-cli` and - `localai-playground-run` are a matched pair. +2. Get the `localai-toolkit--arm64.zip` from this repo's + [releases](https://github.com/ancientcomputing/locallm/releases) (`1.0.0-GA` on) and + unzip it. `localai-cli` and `localai-playground-run` are a matched pair. Older + versions: `1.0.0-beta.4` through `1.0.0-RC.1` are on + [`ancientcomputing/locallm-releases`](https://github.com/ancientcomputing/locallm-releases/releases), + earlier ones in [`../../toolkit/`](../../toolkit/) (see that folder's README). 3. Put both binaries in this folder (or point `LOCALAI_CLI_PATH` at wherever you put `localai-cli` — it looks for `localai-playground-run` in the same directory as itself by default). From a53148f139967758551d9d837bfdfefa7968cffe Mon Sep 17 00:00:00 2001 From: ancientcomputing Date: Mon, 12 Oct 2026 10:59:30 -0700 Subject: [PATCH 73/73] annotated-examples: break up the opening paragraph MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit What the file is, in two sentences; the markers as a table (marker, what it marks, which examples use it — Path A included); why they're comments; and plate-today / plate-today-tools / repo-qa as "two ways to give the model tools". Same facts, shorter paragraphs. Co-Authored-By: Claude --- docs/annotated-examples.md | 53 +++++++++++++++++++++++++------------- 1 file changed, 35 insertions(+), 18 deletions(-) diff --git a/docs/annotated-examples.md b/docs/annotated-examples.md index e950a96..1f98491 100644 --- a/docs/annotated-examples.md +++ b/docs/annotated-examples.md @@ -1,23 +1,40 @@ # Annotated example source -The full source of every reference app, with every line that actually touches the SDK marked -`// ← SDK` (Core), `// ← SDK (Inference)` (the MLX runtime — `code-buddy`, `repo-qa-local`, -`workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, `mlx-control-room`, `mcp-chat`, `jevdk` and `jev-serve`), `// ← SDK (Remote)` (online providers — -`model-switch`, `security-demo` and `jevdk`), `// ← SDK (MCPAppsHost)` (showing MCP servers' interactive views — -`mcp-chat`), or `// ← Components` (`components-demo`, `components-updates-demo`, and `mcp-chat`). Everything else is ordinary SwiftUI/Foundation — the point -of marking it this way is to make obvious just how little of each file is SDK-specific plumbing. -`plate-today` and `plate-today-tools` are a matched pair — the same app twice, "Path B" (hand- -written `Tool` adapters) vs. "Path A" (Core's ready-made ones, `// ← SDK (Path A)`) — meant to be -read back to back; see §7a of `sdk-guide.md` for the framing. `repo-qa` is Path A again, but on -its own, smaller and differently-shaped: a plain CLI tool showing `MCPTool` against a no-auth -server, distinct from `plate-today-tools`' OAuth-gated Todoist case. - -(GitHub-flavored Markdown doesn't apply bold/inline formatting inside fenced code blocks, so a -trailing comment is used instead of `**bold**` — it survives being read as a comment in the real -`.swift` file too, not just rendered here.) - -See [`sdk-guide.md`](sdk-guide.md) for the prose walkthrough these annotate — this file is the -companion "show me the whole thing at once" reference. +The full source of every reference app, with each line that calls the SDK marked by a +trailing comment. Everything unmarked is ordinary SwiftUI and Foundation, so the markers show +how little of each app is SDK code. + +[`sdk-guide.md`](sdk-guide.md) explains the concepts step by step. This file is its companion: +each app shown whole. + +### The markers + +| Marker | What it marks | Used in | +|---|---|---| +| `// ← SDK` | Core | every example | +| `// ← SDK (Inference)` | the MLX runtime, for local open-weight models | `code-buddy`, `repo-qa-local`, `workspace-buddy-local`, `os-matrix`, `aiql`, `vistanova`, `mlx-control-room`, `mcp-chat`, `jevdk`, `jev-serve` | +| `// ← SDK (Remote)` | online providers | `model-switch`, `security-demo`, `jevdk` | +| `// ← SDK (MCPAppsHost)` | showing an MCP server's interactive views | `mcp-chat` | +| `// ← Components` | the prebuilt SwiftUI views | `components-demo`, `components-updates-demo`, `mcp-chat` | +| `// ← SDK (Path A)` | Core's ready-made tools | `plate-today-tools`, `repo-qa` | + +A few markers add a short note in brackets, such as `// ← SDK (type)` for a line that only names +an SDK type. + +The markers are comments, not bold text, because GitHub doesn't format text inside code blocks. +A comment also still makes sense in the real `.swift` file. + +### Two ways to give the model tools + +`plate-today` and `plate-today-tools` are the same app built two ways, so read them back to back: + +- **`plate-today` (Path B)** writes its own `Tool` adapters. +- **`plate-today-tools` (Path A)** uses Core's ready-made tools. + +§7a of [`sdk-guide.md`](sdk-guide.md) explains the two paths. + +`repo-qa` is Path A on its own: a small command-line tool that uses `MCPTool` with an MCP server +that needs no sign-in. Compare `plate-today-tools`, whose Todoist server needs an OAuth sign-in. ## How much code is this, really?