Skip to content

feat(step): load deferred MCP tools from find_tools - #347

Open
lit26 wants to merge 2 commits into
stepfun-ai:mainfrom
lit26:feat/tool-search
Open

lit26 wants to merge 2 commits into
stepfun-ai:mainfrom
lit26:feat/tool-search

Conversation

@lit26

@lit26 lit26 commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

MCP server tools no longer enter the prompt by default; the Step profile's existing find_tools loads the ones a turn needs.

No new tool name is introduced. find_tools already shipped in the Step tool profile and could only see tools that were already active. This change gives it a ranked, activating implementation and an exposure model that leaves tools registered but undeclared.

find_tools before and after

before after
Ranker lowercase includes count per query token Okapi BM25, name-field weighted, exact-name first, then coverage
Pool active tools only (ExtensionContext.getToolCatalog()) deferred tools not yet active, plus active ones
Activation none, it could only describe what was already declared activates the deferred matches; they are declared on the next request
Answer N. name [score=…] + description + schema same numbered format, plus Newly available from your next call: …
Host plumbing getToolCatalog() getActiveTools() / getAllTools() / setActiveTools(); a host with only getToolCatalog() still gets the ranked answer, just without activation

The ranking lives in packages/coding-agent/src/core/tools/tool-search.ts (module name only). The tool_search tool and extension from the first commit were deleted in the second; squash on merge if you prefer a single commit.

Changes

  • ToolDefinition.exposure:
    • direct (default): declared and active at session start.
    • deferred: registered, undeclared, loaded on demand by find_tools.
    • hidden: registered, unreachable, and left out of getAllTools().
    • An explicit selection still wins: a name in defaultTools activates a deferred tool, as does the --tools allowlist.
  • src/step/mcp.ts: server tools publish as deferred by default. New per-server config.toml keys exposure and description.
  • renderMcpServersSection(): a # MCP servers system-prompt section listing deferred servers with their one-line summary. Neither the declared tool list nor find_tools names them, so without this section the model cannot know they exist.
  • ExtensionContext: getActiveTools(), getAllTools(), setActiveTools(), wired through the runner's existing session actions.
  • step/permissions.ts: find_tools is read-only — it changes the session tool set and touches no files.

Review focus

  • The default flip. Deferred is the default for every MCP server, so a model that never calls find_tools loses access to MCP tools. The prompt section is the mitigation. If that trade is too sharp for a default, the alternative is direct by default with per-server opt in — a one-line change in serverExposure().
  • Activation semantics. find_tools only ever adds tools, never removes them, so the additive-activation path that records addedToolNames applies and the change survives resume and fork.

Configuration

[mcp_servers.github]
url = "https://api.githubcopilot.com/mcp/"
description = "GitHub issues and pull requests"
# exposure = "direct"   # small, frequently called server

Existing configs keep working: unknown keys are ignored, and a server without exposure becomes deferred.

Verification

  • pnpm run check clean (biome, boundary checks, tsgo --noEmit, browser smoke).
  • test/tool-search.test.ts — 16 tests: tokenizer and BM25 ordering, activation, deferred/direct/hidden session behavior, settings-based activation, read-only permission decision for find_tools.
  • test/tool-search-integration.test.ts — 2 end-to-end agent turns on the faux provider with the real Step profile: the deferred tool is absent from request 1, declared on request 2, and addedToolNames records the activation.
  • test/tool-search-mcp.test.ts — 2 end-to-end turns against a real streamable-HTTP MCP peer: deferred by default, the prompt section appears, find_tools loads the tool, the model calls it and the server receives the arguments; a direct server is declared without the section.
  • Updated for the behavior change: src/step/mcp-startup.test.ts (+3 exposure and section tests), src/step/mcp-catalog.test.ts (late catalog and removal observed through find_tools), test/step-tool-profile.test.ts (description).
  • 263 related tests green; 128 more across system prompt, extension catalog, and live context.
  • Manual run with a real model and the CLI: pnpm step --approval-mode auto -e ./demo.ts "Find a tool that can look up weather, load it, then call it for Paris." loads weather_lookup through find_tools and answers sunny in Paris, with no settings or config change.
  • A full vitest --run in this checkout fails 30 pre-existing tests that need a real provider credential (No API key found for step); unrelated.

Unit and end-to-end tests use the faux provider; no paid model requests.

Not in this change

  • No codemode scripts and no searchTools(): Step has no codemode surface, so deferred is the only indirect exposure.
  • No per-tool toolExposure patterns (get_*, delete_*): exposure is per server.
  • Native deferred-declaration protocol paths (Anthropic tool_reference, OpenAI tool_search_call) already exist in packages/providers; this change relies on the existing additive-activation plumbing.

A session that registers many tools pays for all of them in every prompt,
even when a turn needs one. Add `tool_search`: a BM25 ranker over tool
metadata that activates the matches so the next model call declares them.
Activation goes through the active tool set, so the change is recorded on
the tool result like any other activation and survives resume and fork.

Tools opt out of the default selection with `defaultActive: false`; they
stay registered and reachable through the tool settings or another tool's
activation. Explicit selection still wins: a name in the `--tools`
allowlist is always activated.

`tool_search` ships registered but undeclared, enabled per session through
`defaultTools`, `--tools`, or the tool settings. Step's permission policy
treats it as read-only: it loads tool definitions and touches no files.
A session that connects MCP servers pays for every server tool in every
prompt. Add tool exposure so a tool can stay registered but undeclared, and
make the Step tool profile's `find_tools` the loader for those tools.

- `ToolDefinition.exposure`: `direct` (default), `deferred` (registered,
  undeclared, loaded on demand), `hidden` (registered, unreachable and left
  out of `getAllTools()`). Explicit selection still wins: a name in the tool
  settings or the `--tools` allowlist activates a deferred tool.
- `find_tools` ranks deferred and active tools with a BM25 ranker and
  activates the deferred matches, which the next request declares. The
  activation is recorded on the tool result, so it survives resume and fork.
- MCP servers publish their tools as `deferred` by default; `exposure` and
  `description` in `config.toml` set that per server.
- The MCP extension adds an `mcp_servers` system-prompt section naming the
  servers whose tools are deferred, because the tool list and `find_tools`
  both stay silent about them.
- `ExtensionContext` exposes the live tool set (`getActiveTools`,
  `getAllTools`, `setActiveTools`) so a profile tool can activate others.

`find_tools` is already classified read-only by the permission policy.
@lit26 lit26 changed the title feat(coding-agent): add tool_search for deferred tool discovery feat(step): load deferred MCP tools from find_tools Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant