Repository navigation
docs: plan goal-centred replan study and intermediate measurements - #6764
Draft
loopx-agent wants to merge 6 commits into
Draft
loopx-agent wants to merge 6 commits into
loopx-agent wants to merge 6 commits into
Conversation
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
…text-study Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal And Delivered Outcome
The original Goal is already present in replan context; presence alone does not prove that an agent reassesses the most consequential remaining gap. This bilingual Draft separates additional whole-goal reflection (G) from a source-bound reading index (C), retaining the original Goal and complete ordinary input in all four cells.
The proposal now specifies an executable research sequence and measurement design: necessary correctness and new late-checkpoint qualification, the primary retained/fresh comparison, stride, G/C, then the lower-priority outcome-refinement candidate. It connects intervention delivery to actual adoption, research evidence, retained candidates, native outcomes and complete costs. It reuses the existing six trajectory judgments, with explicit episode eligibility, denominators, unknowns and analyzer qualification. This is documentation only: no shipped runtime, setting, scoring or experiment behavior changes.
Author Declaration
docs/architecture/rfcs/long-horizon-harness-benchmark-research-program-v0.md.benchmark-toolkitwith the native provider, TypeScript replan/context and trajectory-assessment owners. G/C remain local study labels, with no new protocol or scorer.goal_alignment,benefit_test,discrimination,information_gain,evidence_adoption,redundancy; metric-specific eligibility/window, yes/no/unknown/eligible, N/A and missing-data bounds. All remainuncalibrated_model.Boundary And Remaining Author Work
The disposable prototype invokes the destination CLI once, preserves its current response and identity, and composes only the declared advisory from that response. Its framing/off/error/replay checks are bounded engineering evidence. Matching a few source files, a generated model-visible file, an accepted replan or a stored checkpoint does not prove current client delivery, complete recovery, later adoption or native benefit.
Remaining author work: qualify the actual pinned warm receiver/model-client path; freeze compatible late points, exact treatment inputs, episode boundaries/rubric and matched windows before outcomes; then complete the authorized continuation comparisons and independent assessment. Current-source model transport previously failed before tool use, and an older synthetic denominator remains unqualified. Preserve those failures and costs. Do not borrow an existing fresh arm as an unplanned control, expand the factorial without capacity, reset cooldown/opportunities or add another 36h after restoration.
Keep Draft for this concrete unfinished receiver and empirical work. The documentation revision does not mark the scientific study complete or change product defaults. Review, merge, installation and scientific acceptance remain separate.
Future-facing pass: reuse the typed reader, pre-decision selector, six-judgment vocabulary/key/reducer and existing native/usage owners. Automatic episode discovery, semantic citation entailment, causal adoption links, cross-session full accounting and independent evaluator calibration remain explicit gaps. No speculative production abstraction or parallel authority was added. A later caller-facing option still requires its frontend/CLI/Lark readback and disable journey.
Validation
Documentation head:
fd90ba87bc3cb439f4badb7debf443f492b39724, synchronized with target main55d6337485d4924f1cbb8b41e51a8c02687c332a. The complete PR diff contains only the two study documents and the existing lower-page benchmark-guide link.Frontend / Visual Evidence
UI impact: none. Plain Markdown research planning and an existing lower-page guide link. This increment leaves the opening viewport unchanged. It adds no packaged frontend, installed configuration or browser interaction.
Type Of Change
LoopX Area And Technical Direction
Benchmark research protocol and public docs; existing roadmap S5/S11 and long-horizon measurement/episode acceptance. No new roadmap, authority store, provider migration, runtime routing, setting or shared protocol vocabulary.