Repository navigation
Color F# and C# cells by what their names resolve to - #134
Open
Jonathan-Schaefer-git wants to merge 1 commit into
Open
Jonathan-Schaefer-git wants to merge 1 commit into
Jonathan-Schaefer-git wants to merge 1 commit into
Conversation
A cell editor was colored by Monaco's lexer alone, which sees a type, a function, a module and
a value as the same identifier. Kernels can now classify the symbols in a cell, and the editor
draws those classifications as Monaco semantic tokens, in `verso serve` and in VS Code.
ISemanticTokenProvider is an optional interface beside ILanguageKernel, so no existing kernel
changes. GetSemanticTokensAsync returns tokens typed by the Language Server Protocol's standard
names, which Abstractions holds as constants (SemanticTokenTypes, SemanticTokenModifiers), or
null when the code could not be analyzed, in which case the editor keeps the colors it shows.
Positions are 0-based and relative to the cell, as for a Diagnostic, and a token's Length counts
UTF-16 characters. Verso.Host serves it as kernel/getSemanticTokens, and both notebook services
pass it through; a kernel that does not classify answers empty and is not started for the
request.
The editor asks for diagnostics and for semantic tokens of the same text moments apart, and a
kernel that provides both answers them from one check of that text, so a pause in typing
compiles once. The F# checker keeps its last check, keyed by the text, the compiler options and
the #load'ed scripts with their write times. The C# workspace keeps the last document it built
for a cell's text, and with it the compilation; a run or a new reference drops it.
The F# kernel classifies with FCS's GetSemanticClassification over the cell's own lines, shifted
past the cells executed before it. A dotted name is classified by FCS as a whole, by its last
part, so List.map arrives as one function; each range is cut back to its last identifier, which
leaves List to its own classification as a module. The C# kernel uses Roslyn's Classifier, and
reports a top-level variable, which Roslyn classifies as a field of the script class, as the
variable it reads as.
In the editor, a document semantic tokens provider follows the diagnostics rules: nothing is
asked while a cell runs, a cell is asked again after a run when its diagnostics are, and requests
from all editors go one at a time, so a run in VS Code waits behind at most one. A cell whose
text is unchanged since its answer gets that answer back without a request. The theme builder
colors each standard token type by the syntax color the theme has for it. Verso's light, dark
and high-contrast themes gain an enumMember color for enum members and union cases, and a theme
without one colors them by its type color.
In the VS Code webview the editor takes its syntax colors from the active VS Code theme, and the
semantic token types its lexical rules do not name (class, struct, enum, interface,
typeParameter, parameter, property, enumMember, event, method) would fall to the editor's text
color. Each is colored as VS Code colors it in a source file: by the theme's
semanticTokenColors, laid over its included themes, and the user's
editor.semanticTokenColorCustomizations, where a selector names the type or the type with a
modifier ("variable.mutable", "*.static"), and otherwise by the TextMate scope VS Code falls
back to for that type (entity.name.type.struct, variable.other.enummember, ...). Types the
lexers share keep their lexical rule, which already uses the same scopes. Selectors limited to
a language are not supported. VS Code shows semantic tokens only when
editor.semanticHighlighting.enabled says so or, as by default, when the theme or the user's
customizations do; the notebook follows the same decision, and turns its semantic tokens off
with a theme that has none.
Signed-off-by: Jonathan Schäfer <jonathan.schaefer.mail@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements semantic highlighting (#132)
C# and F# cells were colored by Monaco's lexer alone, which sees a type, a function, a module and a value as the same identifier. Kernels can now classify the symbols in a cell, and the cell editor draws them as semantic tokens.
ISemanticTokenProvideris an optional interface besideILanguageKernel, so no existing kernel changes.GetSemanticTokensAsyncreturnsSemanticTokens, ornullwhen the code could not be analyzed, in which case the editor keeps the colors it shows. Positions are 0-based and cell-relative likeDiagnostic,Lengthis in UTF-16 characters, and the token type and modifier names are constants in Abstractions (SemanticTokenTypes,SemanticTokenModifiers), using the LSP's standard names. Verso.Host serves it askernel/getSemanticTokens.Diagnostics and semantic tokens for the same text share one type check. The F# checker keeps its last check, keyed by the text, the compiler options and the
#loaded files with their write times. The C# workspace keeps the last document it built for a cell's text, and with it the compilation.F# classifies with FCS's
GetSemanticClassificationover the cell's own lines. C# uses Roslyn'sClassifierand reports a top-level variable, which Roslyn sees as a field of the script class, as a variable.Tokens are asked for under the diagnostics rules: never while a cell runs, again after a run together with diagnostics, and one request at a time across editors. A cell whose text is unchanged gets its last answer back without a request.
Each token type takes the theme's color for its kind, so a class takes
typeand a method takesfunction. The built-in themes gain anenumMemberkey, and a theme without one colors enum members and union cases by itstypecolor. In VS Code, tokens are colored as VS Code colors them in a source file: by the theme'ssemanticTokenColorsand the user'seditor.semanticTokenColorCustomizations, and otherwise by the TextMate scope VS Code falls back to. The notebook also followseditor.semanticHighlighting.enabled.I am open to any changes/feedback to this PR, since it originally started as mere eye candy for testing how Marey and Verso would fit together and what UX would be like.