[rhennigan]

Summary

Adds StartMCPSocketServer[server, opts]. It serves an MCPServerObject over MCP's Streamable HTTP transport (the stateless subset, revision 2025-11-25) from inside the current kernel, including a front end's kernel. It listens on 127.0.0.1 with SocketListen and returns an MCPSocketServerObject at once; DeleteObject stops the server. Chatbook's ACP window uses it to give a coding agent (OpenCode) tools that run in the user's kernel.

The user-facing description is in docs/socket-server.md, and the design decisions are in Specs/SocketServer.md.

The socket transport (Kernel/Server/Socket.wl)

• HTTP:
  • One JSON-RPC message per POST, answered as JSON, or as an event stream with keep-alive comments for long calls.
  • 202 for notifications; 405 for GET and DELETE.
  • No Mcp-Session-Id.
  • Its own HTTP/1.1 framing, since SocketListen delivers raw chunks of at most 8 KB.
  • Persistent connections, with responses in request order.
  • Size limits on request heads (64 KB) and bodies (8 MB), and a 30-second timeout for stalled partial requests that doesn't count time the kernel itself was stalled.
• Work in the handler: the SocketListen handler is a preemptive evaluation, so it only frames requests and answers the cheap methods (initialize, ping, tools/list, ...).
• Deferred calls: tools/call and prompts/get are handed to the "EvaluationFunction" as f[run, info]. The caller decides where run[] evaluates the call (for example on the main link), and run[] writes the answer. Every call is answered exactly once, also on aborts, cancellations (notifications/cancelled), rejections, disconnects and internal failures.
• Security:
  • Loopback only.
  • A 256-bit bearer token by default, or a token string, None, or the caller's authentication function. Tokens are compared in constant time.
  • Host and Origin checks against DNS rebinding.
  • Requests are refused on their head, before any body is read.
  • The object and the server records never hold the token; records and jobs keep only its SHA-256 digest.
• Options: "Port", "Path", "Authentication", "EvaluationFunction", "KeepAliveInterval", "ToolOptions".
• Properties: "URL", "Port", "Path", "Token", "JSONConfiguration", "Running", "PendingRequests", and more.
• Isolation: every request and every run[] Blocks all server session globals and $Messages, so nothing a client sends changes the kernel's state.
• Tool options: in a front end kernel, WolframLanguageEvaluator defaults to the "Local" method, so agent code doesn't run in the user's session.
• Paclet reloads: the state of running servers lives in Wolfram`AgentToolsState`SocketServer` , so servers survive a paclet reload.

Changes to shared code

• Kernel/Server/HTTP.wl (new): the request accessors, Accept negotiation, the protocol-version check, JSON/SSE response strings and a new JSON-RPC message classification (jsonRPCMessageType) moved out of Cloud.wl, so the cloud and socket transports share them.

• Cloud transport behavior changes:

  • A message that is neither a request, a notification nor a response now gets a 400 instead of a 202.
  • Only a string or number id makes a request; a request without one is accepted but never evaluated.
  • $clientName is Blocked per request.

• handleMethod["initialize"] no longer sets $Messages = {}, which used to silence the whole kernel for the HTTP transports. The stdio read loop now applies that policy itself (applyStderrPolicy in Local.wl), before it handles the message, as before.

• catchTopIsolated (Common.wl): a catchTop that always starts its own error scope. Preemptive evaluations see the Blocks of the evaluation they interrupt, including another catchTop.

• redactSecrets and withFailureSecrets (Common.wl): internal-failure data, messages, bug reports and log files no longer show secrets. That covers:

  • values of secret keys
  • Authorization: header lines and Bearer tokens
  • all other occurrences of values found that way, if they have at least 8 characters
  • registered secrets, wherever they occur

  The evaluation stack is not searched. The InternalFailures log directory is made private (0700) on Unix-like systems. See docs/error-handling.md.

• Small fix: cleanupOldFailureLogs used Quiet @ DeleteFile /@ files, which parses as (Quiet @ DeleteFile) /@ files and so never quieted anything.

Review fixes (second commit)

A review of the first commit found these; they are fixed and tested:

• initialize advertised the MCP Apps UI extension to UI-capable clients, though the socket transport has UI off for every other request.
• "Authentication" values that can't be functions (False, True, numbers, lists) started a server that refused every request. They now give InvalidSocketServerAuthentication.
• Requests pipelined behind a deferred call were re-scanned in quadratic time when it finished, inside PreemptProtect. They are now read chunk by chunk.
• A maintenance task that can't be submitted now fails the start instead of leaving a server without keep-alives, timeouts or disconnect cleanup.
• Persistent responses now send Connection: keep-alive, which HTTP/1.0 clients need.
• Cleanups:
  • one jobErrorResponse instead of three copies of the same branching
  • a shared internalErrorResponse (in Shared.wl) for the stdio, cloud and socket transports
  • no unused id parameter in dispatchCloudMethod
  • parseRequestBody built on requestBodyJSON

For reviewers

These were raised in the review and deliberately left as they are:

• Default evaluation function: the default "EvaluationFunction" (#1[ ] &) runs tools inside the handler. While a tool runs there are no aborts, no keep-alives and no other requests, and in a front end kernel a tool that calls the front end does so from a preemptive evaluation. Chatbook passes its own function. This is documented as a known limitation.
• Front end call on internal failures: an internal failure in the handler builds a bug report, which in a front end kernel asks the front end for its version. This is documented under Limitations in docs/socket-server.md.
• Redaction scope: redaction applies to every AgentTools internal failure, not only to socket code. So text such as "bearer of …" or an Authorization: line in unrelated failure data is redacted too.
• catchSocket instead of catchMine: the socket entry points, including the exported StartMCPSocketServer, use catchSocket (catchTopIsolated). That breaks the AGENTS.md "use catchMine" rule on purpose, because catchMine would share failure state with an evaluation the handler interrupted.
• 15.0 descriptor leak: 15.0 leaks about four file descriptors per accepted connection that the client closes, so connections are kept alive.

Testing

• Test files (all pass): Tests/SocketServer.wlt (119, with real loopback clients), Tests/CloudDeployment.wlt (179), Tests/InternalFailureFormatting.wlt (45), Tests/StartMCPServer.wlt (94). The rest of the suite was not run.
• CodeInspector: no new issues in Kernel/Server/, Kernel/Common.wl or the new tests.
• Runtime check against the dev paclet in a real kernel:
  • Socket servers, driven over TCP:
    • initialize from a UI-capable client
    • HTTP/1.0 keep-alive
    • a tools/call with the default evaluation function
    • four requests pipelined behind two queued calls, answered in order
    • a wrong token refused with a 401
  • The stdio server, driven over stdin/stdout: initialize, tools/list, ping, and an unknown method.

🤖 Generated with Claude Code