Repository navigation
Judge: say how much of its scope each absence claim was given, and fetch references through MCP - #94
Merged
Merged
Conversation
…tch references through MCP A claim that something did not happen, a verdict of not observed, a question never asked, an assumption never checked, or a person's goal found absent, stays the evaluator's judgment. Beside it asz now reports how much of its scope the evaluator was given, in steps: opened, only listed, not given, and those whose evidence no one can be given. The counts come from what each answer returned, counted once it was given; a navigation log that is not whole makes them unavailable, never smaller. The two divergence causes are renamed question_not_asked and assumption_not_checked. A goal's first finding is its first recorded one, chosen by one function for the close and -show. A person's goal is absent only when every target was placed. A tool call that shares a record with the agent's text reads as its arguments in asz.view. A judge reference can be fetched by one tool of an MCP server, remote over Streamable HTTP or local over standard input and output, speaking revision 2026-07-28 and the four revisions before it. asz judge -reference-tools lists a server's tools. A URL is kept without its query, a server without its path, and a failed fetch as a class and a detail asz builds. The secrets of a fetch are removed from all it keeps, as written and through every JSON spelling. Tests that replace a file write the new file before they remove the old one. On a newer Linux runner, a removed file's inode went to the next file made, so nothing was replaced and the identity checks had nothing to find.
wu-sheng
force-pushed
the
judge-reading-mcp
branch
from
October 28, 2026 00:11
92d9817 to
22bcf3c
Compare
2 of 5 tasks
wu-sheng
added a commit
to apache/skywalking
that referenced
this pull request
Oct 28, 2026
A step on one part of a record reads that part alone, as the AI Sessionizer does since apache/skywalking-ai-sessionizer#94. The document read a tool call that shares a record with the agent's text as that text, and a queued command kept as data beside other text as that text too. Now a tool call reads as its arguments, and a queued command as the prompt inside its own data. On a local Sessionizer root, 10 of 25 LangChain tool calls shared a record with text, and 3 of 109,397 Claude Code ones. No test fixture holds such a record, so the documents the tests compare with the Sessionizer's are unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Say beside each claim of absence how much of its scope the evaluator was given
Four claims of the judge say that something did not happen: a verdict of
not_observed, the causesquestion_not_askedandassumption_not_checked, and a person's goal foundabsent. The record cannotprove any of them, so each stays the evaluator's judgment. A reader still needs to know how far to trust
one: whether the evaluator looked at the place where the thing would have shown.
Beside each claim, asz now reports how much of its scope the evaluator was given, counted in steps:
The scope is the judged work for
not_observedandabsent, and for the two causes the span of onestream from
from, the step where the question or the assumption arose, to the finding's step. Thecounts come from what each answer returned, counted only once it was given: for an API evaluator, when
a later request carried it to the model; for an agent CLI, when its navigation server wrote it to a log
that it closes with a count. A log that is not whole makes the counts unavailable, never smaller.
With it:
clarification_not_soughtisquestion_not_asked, andunsupported_assumptionisassumption_not_checked.-show, ties broken by the order the stages run.absentonly when every target was placed.page and to the evaluators, never as that text.
Fetch references through an MCP server
A judge reference that needs a login, such as a Jira issue, can now be fetched by one tool of an MCP
server: a remote one over Streamable HTTP, or a local one asz starts over standard input and output.
asz speaks the protocol's revision
2026-07-28and the four revisions before it that begin with ahandshake, and offers the server nothing.
asz judge -reference-tools SRClists the server's tools, to choose the tool and its arguments.What asz keeps of a reference goes with the run when it is exported, so it keeps provenance and never a
credential:
it writes anything.
secret one as
[env NAME].error that names the URL.
as written, inside every JSON string literal however it is escaped, and in text escaped as JSON with
no quotes around it, such as a log line, to four levels of string inside string. Which blocks are text
is read before anything is removed. A failed http fetch is cleared of its proxy's credentials.
Tests that replace a file
Five tests that replace a file with the same bytes now write the new file before they remove the old
one. On a newer Linux runner, a removed file's inode went to the next file made, so nothing was
replaced and the identity checks had nothing to find.
Follow-ups
record with text changed in
docs/en/formats/asz-view.md.