diff --git a/docs/proposals/single-editor-hierarchy.md b/docs/proposals/single-editor-hierarchy.md new file mode 100644 index 0000000..658765a --- /dev/null +++ b/docs/proposals/single-editor-hierarchy.md @@ -0,0 +1,231 @@ +# Proposal: one editor for a whole workflow hierarchy + +**Status:** proposal. Nothing here is implemented. +**Affects:** dialogram (most of it), wfpy (a hierarchy export), wfpy-ide (one +profile flag). + +A diagram of a workflow whose nested workflows live in other files opens **a new +editor for every nested workflow** a person drills into. On a deep hierarchy +that is a tab per level, each with its own chat, its own Run button running +something different, and its own layout. This proposes one editor per *root*: +the file that was opened. Every nested workflow under it, in any file, is a view +inside that editor. Three things stay anchored to the root whatever is on +screen: + +1. **the chat session**: the root's, for the whole hierarchy; +2. **the run**: ▶ Run, "Rerun from Here" and the stepper's ⟲ always run the + root workflow; +3. **the layout**: every nested view's layout is the root's, stored per + *instance* in the root file's layout. + +Nested views stay **editable**, in the file that defines them. + +## What happens today + +Line references are to `packages/` unless written out. + +| | nested workflow in the **same** file | nested workflow in **another** file | +|---|---|---| +| double-click | in place: a new `RequestModelAction` to the same client (`diagram-client/src/network-navigation-mouse-listener.ts:163-188`) | **a new editor**: `NavigateToExternalTargetAction` + `cal:openDiagram`, then `vscode.openWith` (`extension-core/src/extension/diagram/glsp-activation.ts:683`) | +| breadcrumb / "Used By" | in place | opens or focuses the other editor (`diagram-client/src/navigation-ui.ts:524-534, 835-845`) | +| chat | the editor document's (`chat-panel-integrated.ts:328-343` → `glsp-chat-transport.ts:77-92`) | the nested file's own session | +| ▶ Run | **the workflow on screen** (`__calDiagramContext`, `navigation-ui.ts:313-323`) | the nested file's workflow, standalone | +| layout | `.layout/.layout.json`, keyed by workflow name (`diagram-server/src/server/source-model-storage.ts:443-458`) | the same: owned by the defining file, shared by every parent and instance | + +No setting keeps a cross-file drill-down in place. `graphSourceNavigation` only +adds fallback args, and wfpy turns it off. + +**What already works in favour of one editor:** + +- **The server renders whatever file a request names.** `loadSourceModel` takes + `options.sourceUri` with no check against the editor's document, and wfpy's + `cli-plan` source runs `wfpy plan --workflow `. A request + from the root editor's client for `other.py` already renders `other.py`. +- **Edits already go to the file on screen.** Edit handlers write through + `modelState.sourceUri`, which follows the last request. Each + `ReversibleWorkspaceEditCommand` captures its own `uri`, so one undo stack + across files undoes each edit in its own file. +- **The chat already stays on the root.** It is keyed by `diagramIdentifier.uri`, + the editor's document, which does not change when the model does. +- **Overlays already reach a nested view through the trail.** `wf:navTrail` + (`[{sourceUri, workflowName, workflowInstanceName?}]`) makes the overlay lookup + accept the root run (`source-model-storage.ts:940-984, 2281`) and maps the + active node by instance path (`sidecar-toolkit/src/server/source-analysis.ts:806-866`). + +## The design: the editor is the root, and every view is a view of it + +### The root and the trail + +An editor's **root** is its document and the workflow first shown in it: +`trail[0]`. Every model request the editor makes carries the **full trail** from +the root to the view: drill-downs, breadcrumbs, refreshes, and the host's own +refreshes after a save or a run. The trail is what tells the server which file +to render, what tells the overlay lookup which run to show, and (below) what +keys the layout. + +The host keeps the current trail **per editor**, not per shown file. Today +`uriToRefreshContext`, the pending/recent navigation maps and +`requestRunRefresh`'s client lookup are keyed by the *shown* file's URI. An +editor whose root is `top.py`, showing `child.py`, has to refresh as "`top.py`'s +editor, at this trail". + +### 1. Navigating in place + +A drill-down into a workflow defined elsewhere sends a `RequestModelAction` to +the **same** client, with `sourceUri` = the defining file and the trail extended +by the instance. The breadcrumb and "Used By" do the same for any crumb, in any +file. "Same file" stops being a question. `editorContext.sourceUri` (the root) +and the shown file are different things, and navigation never compares them. + +**Open in Its Own Editor** stays available as a node context-menu item. It +opens the defining file as its own root, today's behaviour on request. "Used By" +navigates *up*, out of the root, so it keeps opening the caller as a new root in +its own editor. Only the views *below* the root are in place. + +This changes how every product using the platform navigates, so it is a profile +choice: `nestedNavigation: 'in-place' | 'new-editor'`, default `'new-editor'` +(today's behaviour, so streamblocks-ide is unaffected). wfpy-ide sets +`'in-place'`. + +### 2. The chat stays the root's + +Already true. It is keyed by the editor's document. Two gaps close: + +- **What is on screen.** Each turn's context gains the trail (*viewing + `root › encoder(enc) › block(b2)`, defined in `layers/block.py`*). Selected + node ids are then unambiguous: they belong to that instance. +- **The in-host GLSP-MCP tools** read `modelState.sourceUri`, which follows the + view. Tools that act on the diagram keep acting on the view. Tools that answer + "which workflow is this chat about" read the root. + +### 3. The run is the root's + +`__calDiagramContext` gains `rootSourceUri` and `rootWorkflowName` (trail[0]) +beside the shown ones. ▶ Run, "Rerun from Here" and the stepper's ⟲ use the +root. The run driver's out directory, live-overlay watch and post-run refresh +are keyed by the editor (the root), so a run started while viewing `block` +refreshes `block`'s view in that editor. + +The overlays of a nested view already come from the root run through the trail. +That keeps working once every request carries the full trail. A nested view +**without** a trail (opened in its own editor on purpose) keeps finding runs of +its own workflow, as today. + +### 4. Editing a nested view + +Editable, in the file that defines it, which the edit handlers already do. What +the host has to add: + +- **Secondary documents never stay dirty.** The sidecar writes an edit straight + to disk. Undo and redo instead apply a `WorkspaceEdit` to the file's + `TextDocument` and leave it unsaved. That is fine for the root, whose custom + editor saves it, and wrong for a nested file: the next sidecar edit and the + next `wfpy plan` read the stale file on disk. After an undo or redo applies to + a document other than the root, the host saves that document. The rule is + simple: a file shown inside another file's editor is never left dirty. +- **The root editor refreshes when any file in its trail changes.** Today the + external-change and live-preview refreshes fire only for a document that has + its own client. The root editor watches every file in its current trail, and + refreshes its view at the trail when one changes, whether by an edit made here, + in a text editor, or by git. +- **Concurrent editors.** The same nested file may also be open in its own + editor ("Open in its own editor", or because someone opened it directly). Both + edit through the sidecar's optimistic concurrency (content-hash revisions), so + a stale edit is refused rather than lost, and each editor refreshes from disk. + +### 5. Layout per instance, owned by the root + +A nested view's layout key becomes the **instance path from the root**: +`rootWorkflow/encoder/block/...`. It is stored in the **root file's** layout file +(`.layout/.layout.json`). The hook exists +(`source-model-storage.ts:443-458` qualifies `networkId` with `cal:rootWorkflow` +and `cal:instancePath`) and is off for wfpy, and the layout file path still +follows the shown file. Both change for an in-place view: path from the root, +key from the trail. + +Two instances of one workflow can then be arranged differently. A view with no +saved layout yet starts from the **defining file's** layout for that workflow, +which is today's key, so every layout made so far carries over. It is saved +under the instance key from then on. + +### 6. Rendering a nested workflow as its instance + +wfpy renders a nested view standalone (`wfpy plan child.py --workflow block`), +not as the instance it is. That mostly works, because the overlay maps the +active node by instance path. It goes wrong where the instance matters: + +- **edges and queue-trace entries** are matched by endpoints, not instance + path, so a workflow used twice can show the other instance's tokens and queue + sizes; +- **parameters** given to the instance (a factory's arguments, `instance=` of a + workflow instance) are not those of a standalone build. + +wfpy exports the whole hierarchy instead (see the outline below), each nested +graph elaborated as the instance it is and carrying its instance path, so the +platform asks for the view at a trail rather than a workflow by name. + +### 7. The hierarchy outline + +Modelled on mlir-viewer's file outline (`mlir-viewer/packages/client/src/outline-panel.ts`, +`outline-tree.ts`), which solves the same problem for large `.mlir` files. + +- **The tree is plain data on the root model's args**: `wf:hierarchy`, JSON. + Nothing is laid out until a row is opened, so the panel costs nothing on a + large hierarchy. It is computed from the **unfiltered** hierarchy, so a filter + never hides the way out. +- **A row per instance**: its instance name, the workflow it instantiates, the + file that defines it (dimmed when it is not the root's), and its size (nodes, + nested instances). During and after a run a row also carries the run's state: + running, done, failed, and fire counts, from the overlay already loaded. +- **Single click is a look**: select and center the instance's node in the view + that contains it. **Double click commits**: navigate in place to it (a request + at its trail). The breadcrumb and the outline are the same trail, so the + current view's row is highlighted and its ancestors opened. +- **Filter** keeps the ancestors of a match (`outlineMatches` in mlir-viewer), + since the tree is the only way to reach a deep instance. +- **Chrome**: a collapsible side panel with a toggle button and a key (`O`), as + in mlir-viewer. Its pure helpers live in a GLSP-free module, so they are unit + tested headlessly. + +**One export for the whole hierarchy, cached.** Today each drill-down starts a +`wfpy plan` process for one workflow. `wfpy plan --format graph --hierarchy` +exports the root and every nested graph once (wfpy's +`RewriteEngine.export_workflow_graph` already resolves nested workflows across +imports, with recursion guards). The server caches it per root and invalidates +it when any file in it changes. A drill-down then reads the cache. The outline +comes from the same export, and is the cheap part of it: names, files and sizes. + +## Phasing + +1. **In-place navigation.** The profile flag, drill-down / breadcrumb / "Used + By" in place, the host's per-editor trail and refresh context, and "Open in + its own editor". +2. **The root runs, the chat knows the view.** Root fields in the diagram + context, and Run, Rerun from Here and ⟲ from the root. The run driver and + refresh keyed by the editor. The trail in the chat's turn context, and the + GLSP-MCP root/view split. +3. **Editing nested views.** Secondary documents saved after undo/redo, the root + editor watching every file in its trail, and the concurrency behaviour + tested. +4. **Layout per instance**, in the root's layout file, falling back to the + defining file's. +5. **The hierarchy export, the cache and the outline.** wfpy `--hierarchy`, + instance-elaborated nested graphs, the server cache, then the outline panel. + +Each phase is usable on its own. 1 and 2 together are what removes the +editor-per-level problem. + +## Decided + +- **Nested views are editable**, in their own files (section 4). +- **Layout is per instance**, in the root's layout file (section 5). +- **"Open in its own editor" opens the nested file as its own root.** It gets + its own chat, its own runs and its standalone layout: today's behaviour, made + explicit. A second editor on the same root is not offered. +- **"Used By" opens the caller as a new root**, in its own editor. This editor + keeps its root, chat and run: navigating up never re-anchors it. + +## Open questions + +- **How large is the cached hierarchy** for the biggest real workflows, and + does the outline need the node budget mlir-viewer's has (`truncated`)?