Skip to content

Add opt-in Pyright language server for Python - #168

Open
Grammy Jiang (grammy-jiang) wants to merge 8 commits into
microsoft:mainfrom
grammy-jiang:feat/pyright-language-server-20261021
Open

Grammy Jiang (grammy-jiang) wants to merge 8 commits into
microsoft:mainfrom
grammy-jiang:feat/pyright-language-server-20261021

Conversation

@grammy-jiang

@grammy-jiang Grammy Jiang (grammy-jiang) commented Oct 21, 2026 •

Copy link
Copy Markdown

Summary

Introduce Pyright as an opt-in Python language server for multilspy, preserving Jedi as the default. Follow existing LanguageServer, LanguageServerHandler, LSP initialization and adapter conventions; fix protocol response shapes shared by existing language backends.

Current contributor revision: 9203732e4e5f65cc1374b109110f7b6354d3d43b (JSON-RPC request lifecycle fix over previously verified b5325f1). Real Pyright version pinned for tests: 1.1.413 (pyright-langserver --stdio). This is an upstream library contribution; it is not a production security sandbox or a Binnacle MCP release.

Features and how they are accessed

Pyright 1.1.413 feature multilspy access Validation
Definition, references, hover High-level request_definition, request_references, request_hover Real server across files, inheritance, null responses and UTF-16
Completion, completion item resolve High-level request_completions and low-level resolve Real incomplete-completion retries; label-only and stable-order regressions
Document and workspace symbols High-level requests Real server symbols and workspace indexing
Declaration, type definition, document highlights server.send.* Exact low-level response fixtures
Signature help, incoming/outgoing call hierarchy server.send.* Exact low-level fixtures against real Pyright
Prepare rename, rename server.send.prepare_rename, server.send.rename Cross-file WorkspaceEdit proposal; no automatic disk modification
Code actions / Quick Fix server.send.code_action Real Create Type Stub quick fix from an isolated untyped dependency
Organize imports server.send.execute_command + optional editor-side handler Real Pyright command and authorized workspace/applyEdit applied to a disposable file
Create Type Stub server.send.execute_command Real command generates .pyi files directly in a disposable environment; does not need workspace/applyEdit
Push diagnostics, four type-check modes Async PyrightServer.next_diagnostics() Real off/basic/standard/strict modes, finite deadlines and bounded queue
Synchronous facade SyncLanguageServer Real definition, references, completions, hover and cleanup

The adapter loads a packaged initialize_params.json, explicitly chooses UTF-16, checks essential server capabilities and tears down the child process after successful exit, initialization failure, cancellation or graceful-shutdown timeout. Config appends the python_server option so older positional callers and Jedi defaults remain unchanged.

Workspace writes and security boundaries

  • Default behavior: the adapter declines server-initiated workspace/applyEdit and disables organize-imports execution. It does not apply returned rename proposals.
  • An embedding editor can opt in by setting PyrightServer.set_workspace_edit_handler(async_callback, timeout_seconds=...) before startup. The adapter advertises edit support only if configured, validates LSP ApplyWorkspaceEditResult, restricts the callback lifetime, bounds its execution time, and reports failures without echoing exception paths. The caller remains responsible for edit authorization, document versions, changes and documentChanges, atomic application, filesystem operations and synchronizing any open document buffers through didChange.
  • Important: This callback is not an OS-level sandbox. Pyright 1.1.413's raw pyright.createtypestub command can write .pyi files directly, without asking the client to apply a WorkspaceEdit. Never pass unrestricted command execution to untrusted callers. Process/workspace permissions must be enforced by the embedding application.

Testing and CI

  • Native Windows GitHub Actions, Python 3.10, 3.12, 3.13, 3.14: 105/105 passed on each version at current 9203732, including actual Pyright, Unicode paths, Windows URI spelling, editor callbacks, process lifecycle, and shared JSON-RPC cleanup regressions.
  • Native macOS ARM64, Python 3.10, 3.13, 3.14: 152/152 targeted tests passed per version at 9203732, including real Pyright 1.1.413, Jedi compatibility, edit proposals, path and request/process lifecycle regressions.
  • Native Raspberry Pi Linux ARM64, Python 3.10, 3.13, 3.14: 152/152 targeted tests passed per version, from a patch verified byte-identical to the macOS version. Historical b5325f1 passed 126 targeted tests per host before this shared lifecycle fix.
  • Fork-hosted Ubuntu full-suite CI on Python 3.10, 3.12, 3.14: all three jobs SUCCESS at 9203732, each 197 passed, 16 skipped, 1 xfailed, 2 xpassed. The skipped tests are existing environment-dependent upstream cases, not hidden Pyright binary/version mismatches. Real Pyright 1.1.413 is installed and verified during CI.
  • Focused Ruff checks, source compilation, YAML validation, wheel build and wheel resource inclusion have been checked. Python fixture paths are disposable; no production or user source files are edited by these tests.

Historical b5325f1 fork verification: native Windows four-version matrix + Linux full suite. The new 9203732 run is linked above. These fork CI results are independent of Microsoft-hosted workflow approval.

Shared JSON-RPC request lifecycle fix (9203732, 2026-10-22)

Follow-up from an independent Binnacle integration acceptance run against the real patched Pyright adapter found that a timed-out/cancelled LanguageServerHandler.send_request() retained an entry in _response_handlers even after the provider subprocess stopped. This reproduced natively on macOS and Raspberry Pi Linux. The original failure was preserved as an explicit private Binnacle release gate; with this patch it turns green (37/37 related real-provider/policy tests on each tested host).

The fix lives in multilspy's shared lsp_protocol_handler/server.py, not in the Binnacle server or a second Pyright adapter. It now cleans requests in finally, uses predicate-based condition waits (no lost wakeup), wakes pending callers on EOF/stop, drops late/duplicate replies safely, rejects malformed reply envelopes, serializes concurrent stdin writes/drains across Python versions, isolates prior-reader cleanup from a restarted session, and releases completed background tasks. Cancellation continues to propagate rather than being swallowed. Tests cover 100 repeated cancellations, 20 concurrent out-of-order results, blocked writes, malformed and incomplete frames, crash/EOF, restart, errors, null results and actual Pyright requests after cancellation.

Contributor native targeted regressions, including real Pyright 1.1.413: 152/152 each on macOS ARM64 Python 3.10/3.13/3.14 and Linux ARM64 Python 3.10/3.13/3.14. Ruff, source compilation, wheel build/resource inclusion, and macOS-vs-Linux patch hashes also passed. An independent read-only code review was performed and its substantive concurrency/race findings were addressed with tests.

Current revision fork CI: 7/7 jobs SUCCESS for 9203732 — 105/105 native Windows targeted tests per Python 3.10/3.12/3.13/3.14 and Ubuntu full-suite 197 passed per Python 3.10/3.12/3.14. The previous 7/7 fork CI belongs to historical b5325f1. Microsoft-hosted Test Multilspy and CodeQL currently remain action_required until authorized maintainers approve them; human review and merge are also external gates.

Limits / maintained compatibility

Features not advertised to this specific client profile by Pyright 1.1.413 (such as implementationProvider, selected formatting and pull-diagnostics features) are not claimed; LSP MethodNotFound behavior is tested. All non-automatically-applied edit results remain proposals until the embedding client handles them. A newly released Pyright version must be revalidated against the version-pinned capability and wire-format tests.

Shared LSP fixes include optional originSelectionRange handling in LocationLink, nullable references, LSP-valid completion items, bounded incomplete-completion retry, UTF-16 document buffer indexing, stable completion candidate ordering, and Windows drive-case portability. The changes are covered by the Python backend and shared cross-adapter regressions.

Related: historical basedpyright proposal #39. This contribution targets official pyright-langserver and current multilspy main (baseline 0.1.1), not basedpyright or a privately maintained fork.

Upstream release status: pending Microsoft human review and merge. Microsoft-hosted CI approvals are controlled by repository maintainers; fork CI is reported separately and is not presented as Microsoft acceptance.

Copilot AI balanced review requested due to automatic review settings October 21, 2026 12:08

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

It adds a new external language-server integration whose real runtime behavior is only covered by an environment-gated test and modifies shared definition/reference parsing used by every backend, so human verification is warranted.

No open issues 🎉

What changed in this PR

This PR adds an opt-in Pyright language server backend for Python to multilspy, keeping Jedi as the default. It introduces a PyrightServer adapter that reuses multilspy's existing LanguageServer/LanguageServerHandler infrastructure (launching pyright-langserver --stdio), adds a python_server config option to select the backend, and makes two small, backward-compatible robustness fixes to shared response parsing so that LSP-valid results from Pyright (and other servers) are handled correctly.

Changes:

  • Add python_server config ("jedi" default or "pyright") and route it in the LanguageServer.create() factory, raising MultilspyException for unknown values.
  • Add PyrightServer adapter with read-only workspace/configuration/applyEdit handlers, bounded publishDiagnostics delivery, UTF-16 initialization, and shutdown/stop lifecycle handling.
  • Relax request_definition to accept LocationLink results without the optional originSelectionRange, and treat a null references response as an empty list; add synthetic + env-gated integration tests and README docs.
File Description
src/​multilspy/​multilspy_config.py Adds python_server Literal field (defaults to "jedi").
src/​multilspy/​language_server.py Routes Pyright in the factory; relaxes LocationLink parsing and guards None references.
src/​multilspy/​language_servers/​pyright_language_server/​pyright_server.py New Pyright adapter with init, diagnostics queue, and lifecycle handling.
src/​multilspy/​language_servers/​pyright_language_server/​__init__.py New empty package init.
tests/​multilspy/​test_multilspy_pyright.py Synthetic backend-selection/parsing tests plus env-gated real Pyright test.
README.md Documents Pyright as an optional Python backend and how to select it.

Review effort: Balanced

@grammy-jiang

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

… request shapes

Add a binary version check, an advertised-capability matrix test, one
data-driven corpus of exact low-level payloads (declaration, type
definition, highlights, prepare rename, signature help, call hierarchy,
UTF-16 non-BMP columns, null results, code action, rename proposal,
completion resolve, MethodNotFound for unadvertised requests), and push
diagnostics for every typeCheckingMode. Document the capability matrix
in the README. 74 passed on Linux ARM64 with real Pyright 1.1.413.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@grammy-jiang
Grammy Jiang (grammy-jiang) marked this pull request as draft October 21, 2026 13:57
@grammy-jiang
Grammy Jiang (grammy-jiang) marked this pull request as ready for review October 21, 2026 14:20
…rage

Run pinned real Pyright in CI, preserve latest diagnostics across queue overflow, reset stale state on restart, validate advertised capabilities and bound shutdown.

Correct shared UTF-16 document edits and stable completion ranking; add Pyright/Jedi protocol, Unicode, Windows CLI and lifecycle regressions across Python 3.10-3.14.
Support explicit, bounded workspace/applyEdit authorization with fail-closed LSP response validation. Cover genuine organize-import edits, create-type-stub direct writes, and documented process/workspace isolation boundaries.

Fix Python 3.14 Windows file URI drive casing and strengthen portable LSP tests. Verify pinned Pyright 1.1.413 across Windows 3.10/3.12/3.13/3.14, Ubuntu full suite 3.10/3.12/3.14, and native macOS/Linux regression suites.
@grammy-jiang

Copy link
Copy Markdown
Author

Maintainer review requested for the finalized Pyright adapter at b5325f19c7b77317bd4ab7c1ff3c6bb8fa5b8f1b. Independent fork CI is green in all 7 jobs: native Windows Python 3.10/3.12/3.13/3.14 (104 tests passed on each) and full Ubuntu multilspy suites on Python 3.10/3.12/3.14 (171 passed on each, with existing skips/xfail). See https://github.com/grammy-jiang/multilspy/actions/runs/39660495843 . The PR description now documents supported Pyright 1.1.413 capabilities, opt-in workspace edits, direct type-stub-writing command behavior, and OS sandbox responsibilities. The Microsoft-hosted Test Multilspy and CodeQL workflows currently show action_required pending a maintainer approval; could a maintainer approve those workflow runs and review the PR? Thank you.

Clear pending requests on cancellation, stop and EOF; ignore late, duplicate and malformed replies. Serialize concurrent stdin drains for Python 3.10/3.11, fence stale readers after restart, and reap finished dispatch tasks.

Add concurrent, stalled-writer, late-response, malformed-JSON-RPC, interrupted-frame, process-crash and real Pyright restart regression tests across Mac and Linux.
@grammy-jiang

Grammy Jiang (grammy-jiang) commented Oct 21, 2026 •

Copy link
Copy Markdown
Author

Follow-up fix pushed on the existing PR #168 branch: 9203732 (fix(lsp): release cancelled requests and serialize lifecycle IO). A native Binnacle integration test found that a cancelled/timed-out JSON-RPC request remained registered even after the Pyright provider stopped. This commit fixes pending-request cleanup/wakeup and late/duplicate replies in shared multilspy, serializes stdin drain for older Python versions, guards stale readers across restarts, and reaps completed dispatch tasks. The previously failing Binnacle request-cleanup gate now passes without a Binnacle-local adapter or workaround. Contributor native targeted tests: 152 passed per environment on macOS ARM64 and Linux ARM64 (Python 3.10/3.13/3.14). Fork verification: https://github.com/grammy-jiang/multilspy/actions/runs/39690294398 — all seven fork CI jobs are successful: Windows native Python 3.10/3.12/3.13/3.14 each 105 tests passed; Ubuntu full multilspy suites Python 3.10/3.12/3.14 each 197 passed, 16 skipped, 1 xfailed, 2 xpassed. These are verified fork-hosted results on the exact PR head 9203732, not Microsoft-hosted workflow approval. The PR description has been updated to distinguish the current revision from earlier CI evidence. Maintainer review and Microsoft-hosted workflow approval remain requested. Thank you.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants