Skip to content

PT-4201: Standard-view copy/paste USFM fidelity - #13

Merged
tjcouch-sil merged 44 commits into
mainfrom
standard-view-pt-4201
Sep 23, 2026
Merged

tjcouch-sil merged 44 commits into
mainfrom
standard-view-pt-4201

Conversation

@tjcouch-sil

@tjcouch-sil tjcouch-sil commented Sep 15, 2026 •

Copy link
Copy Markdown
Member

Implements PT-4201 (WI-14): copy/paste USFM fidelity in Standard view.

Moved from eten-tech-foundation/scripture-editors#532, following docs/transferring-work-from-eten-tech-foundation.md, so this work lands where Paratext 10 now develops platform-editor. Rebased with --onto origin/main from the old base (768f6476, the #531 annotations fix), so only this branch's own 19 commits replayed — clean, no conflicts, and verified to have carried every PT-4201 change intact.

Two things reviewers should know about the move:

  1. The committed packages/*/dist is rebuilt here (this repo commits it, eten-tech-foundation does not, so the inherited dist predated every commit on the branch). node scripts/verify-committed-dist.mjs passes.
  2. This repo's main does not have eten-tech-foundation's #531 ("apply annotations issued while a load is in flight"). That is why the suite reports fewer tests here than the old PR did — platform-editor 1850 vs 1864, shared-react 1668 vs 1674. Nothing from PT-4201 was lost; the delta is entirely use-load-gate.hook, LoadStatePlugin and the annotation-before-load tests, none of which this branch touches. Transferring #531 is a separate main-sync decision.

What this fixes

Verified against the current main, which still had none of these fixes.

Copy

  • text/plain is byte-faithful USFM: note callers emitted (were silently dropped and rewritten to + on re-paste), no phantom separator spaces, AttributeRunNode-transparent.
  • A copy with nothing selected no longer writes a # to the clipboard. @lexical/clipboard's execCommand hack injects a hidden <span>#</span> and bails before preventDefault() on an empty selection, so Chromium copied the placeholder. Reported for \fig, but it fired for a plain collapsed caret in every view and in scribe too, and for cut.
  • Opaque constructs survive the internal clipboard flavor. excludeFromCopy now keys on child count rather than tag: previously a copied figure/sidebar/periph/ref pasted its full literal USFM bytes on screen while the USJ silently dropped the node and its attributes.
  • Sidebars and tables copy on one line so a per-paragraph rebuild can reassemble them.

Paste

  • Positional NBSP rule (line-start and next-to-a-marker → space, interior → ~) replacing the blanket mapping that produced \f~ \fr~1:1 … unknown-marker soup. No pasted NBSP is ever dropped (see the 2026-09-18 review round).
  • \c/\id stripped anywhere in a pasted line — a pasted \c used to poison the save loop ("Multiple chapter markers present") with the error only in the renderer log.
  • Attribute-run pastes route as plain text, byte-matching typed input.
  • A pasted \fig's caption is no longer removed — a shared text-spacing transform ejected the freshly-materialized caption out of its wrapper. Only reproducible on a full Editor mount, which is why engine-level tests never saw it. The misplaced gray box was the same defect. It was also silently breaking \ref pastes.
  • A paste's provenance now survives a deferred Tier-2 settle, so a pended rebuild keeps the own-prefix dedup (typed input keeps its existing behavior, pinned both ways). Removed in the code-review round below — the dedup it carried was deleting the host paragraph's marker. See "Paste no longer edits the host paragraph's marker".

Review follow-ups (this round)

  • UnknownNode.isSelected dropped the node-selection case. The override answered purely on whether any of the node's own children were in selection.getNodes(), which is the right test for the range boundary it was written for but is never satisfied by a NodeSelection — that marks a node by its own key and puts no children in the list. A construct selected outright therefore reported unselected, and $appendNodesToJSON hoists an excluded wrapper's (also unselected) children in its place, so the copy would have carried nothing for it. The base key-membership answer now runs first for a node selection; the child-membership rule is unchanged for everything else. Not reachable from today's UI — both $createNodeSelection sites target verse markers — but it is a trap for whoever adds node-selection over figures or tables.
  • ContextMenuPlugin depended on ClipboardPlugin for its empty-copy guard. Both shipped editors mount the pair, but the plugin is exported on its own, and a host mounting it alone got exactly the placeholder-overwrites-clipboard bug the guard exists to prevent. It now registers the guard itself; a duplicate registration at the same priority is inert (the first listener to claim an empty selection stops propagation).

Both are covered by tests that go red when the fix is reverted — including a ContextMenuPlugin-without-ClipboardPlugin mount.

Verification

Full workspace green (--skip-nx-cache): platform-editor 1828, shared 683, shared-react 1666, utilities 56; typecheck and lint clean. Semantics documented in docs/clipboard-semantics.md, with accepted asymmetries and remaining deferrals listed explicitly.

Re-verified after the follow-ups above: shared 697, shared-react 1674 (+1 skipped), platform-editor 1857 (+4 skipped); nx format:check, lint and typecheck clean.

The companion e2e spec runs and passes through the real OS clipboard (paranext-core #2656), against a build of this branch.

Every fix here was reviewed by reverting it and confirming the pins go red. Two review rounds caught defects in my own earlier attempts, including one that would have shipped silent data loss.

AI-assisted — Claude Code (Opus 5). Task ledger and per-task reports in .superpowers/sdd/ (untracked).

🤖 Generated with Claude Code

https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f

Hand-QA follow-ups (2026-09-16/17)

Four findings from the first hand-QA pass (ddea1549 + 40d06486, a1dede31, dist 44d69dbd/e91c930c):

  • A drag that starts on a read-only glyph, or runs from a figure's caption into its attribute run, copied nothing / stopped short. Two layers. (1) Lexical 0.43's $internalResolveSelectionPoint resolves a DOM point inside a decorator's element to null, nulls the whole selection, and the reconciler removeAllRanges() — a \fig's \fig / |src="…" / \fig* glyphs are ImmutableTypedTextNode decorators whose text is a DOM child of the decorator, so a drag whose focus entered the attribute run collapsed. New DecoratorBoundarySelectionPlugin (shared-react, mounted in every view) handles SELECTION_CHANGE_COMMAND at CRITICAL and gives Lexical the selection it refused: each offending end snapped to the decorator's boundary in its parent (a range grows outward so the glyph is wholly in; a caret takes the nearer side), built via $createRangeSelectionFromDom from a selection-shaped object. (2) Measured in the Electron app: Chromium extends a mouse drag only from a base it placed itself, so the plugin must NOT rewrite the DOM selection while a button is down — it marks the snapped selection clean during a drag and materializes it into the DOM on pointerup/pointercancel (an event type Lexical does not re-derive from the DOM), and writes immediately for keyboard/programmatic arrivals. Chromium also clamps any drag begun inside a contenteditable="false" island to that island, so a drag from the \fig glyph selects at most the whole figure — the browser's rule, recorded in the comments and the e2e. The note caller is excluded (its landings belong to TrailingNoteCaretGuardPlugin; measured conflict, pinned).
  • P10 → P9: a pasted note had no caller. P9 paste is HTML-first (IHtmlEditorCopyPaste.cs, then Standard-Reverse.xslt reads the fragment's text() as USFM). Standard view's text/html was Lexical's DOM export, which carries a collapsed note's caller only as data-caller and omits every UnknownNode (exportDOM → {element: null}), so P9 received \x \xo …\x*. text/html is now the same USFM bytes as text/plain (usfmToClipboardHtml; invertDisplayNbspInHtml removed with the export it normalized). Asymmetries Flaky interaction tests: ScriptureReferencePlugin and marker-palette each failed once in CI #3/Consider committing built dist/ like platform-bible-react, so consumers stop building at install time #4 in docs/clipboard-semantics.md are closed by construction.
  • P9 → P10: a pasted footnote was the literal caller glyph. P9's text/plain is the visible text (a note = its caller), while the note's bytes ride the html as an escaped <!--usfm:…--> comment inside an exclude span. New paratext9HtmlToUsfm ports Standard-Reverse.xslt over the DOM and wins over text/plain whenever the html carries P9's signature (a usfm: comment, or usfm_<name> + usfmopen/usfmclosed — nothing P10/Word/Docs writes).
  • Simple mode: pasting a \fig did nothing — not intentional. Under structure protection the Standard-view claim declined every paste to StructureKeyboardPlugin's html sanitizer, whose carrier could not express a figure; the same decline also skipped the \c/\id strip (a protected paste of \c 7 created a second chapter node — the save-loop poisoning the strip exists to prevent) and the NBSP rule, and would have skipped the P9 decode. The engine has no protection gate for marker bytes (typed or pasted \v/\p tokenize identically in both modes), so the sanitizer only ever guarded DOM verse/para nodes that Standard view's html no longer produces. The claim now handles a protected document with the same bytes as an unprotected one, except it declines a selection $shouldBlockSelectionReplacement refuses (one owner for that rule) and never replays paragraphs (lines join with a single space). Documented in S3.

Pins: decoratorBoundarySelection.test.tsx (11 — DOM untouched while the button is down, materialized on release, written immediately otherwise, survives an event-less update), DecoratorBoundarySelectionPlugin.test.tsx (2), paratext9Clipboard.utils.test.ts (18), html/plain parity in clipboardCopyFidelity, P9→P10 in markerPasteFidelity, protectedFigurePaste.test.tsx (12), the rewritten structure-protection describe in whitespaceDisplay.plugin.utils.test.tsx (5; the Editor→plugin wiring pin stays falsifiable as a mode difference). Full suites: shared 697, shared-react 1670 (+1 skipped), platform-editor 1903 (+4 skipped); lint, prettier and per-package typecheck clean; verify-committed-dist ✓.

Known, deliberately left (pre-existing, reported to TJ): in Simple-mode Standard view a paste with the caret parked at the end of a \v glyph is refused outright by $selectionContainsVerseMarker (a text-point caret inside the mutable VerseNode counts as containment).

Real-browser verification: paranext-core's clipboard-usfm-round-trip isolated e2e, extended with the four drag/flavour/P9 steps, passes against dist e91c930c linked into the app — 1 passed (2.4 min), first attempt.

Code-review round (2026-09-17)

A /code-review pass over both PRs, then triage. Three commits: 24b030d2 (review fixes), a0efe7df (periph), 1b2163ac (paste behaviour).

Paste no longer edits the host paragraph's marker — behaviour change, please read

Pasting a whole-paragraph copy at an existing paragraph's content start was deleting the host paragraph's marker. A whole-paragraph copy carries its own \p literal, so the line ends up with two paragraph markers, and $withoutRedundantOwnPrefix resolved that by dropping the host's glyph: paste \q1 asdf at \p ‖fdsa and you got one \q1 asdffdsa paragraph, with the \p gone and content that belonged to it now under \q1. An \s1 heading pasted into came back as a plain \p.

That was never a product decision — it was added to un-skip a round-trip test whose target document is an empty \p scaffold, and the "stray empty paragraph" it removed was that scaffold. It is also not what Paratext 9 does: NormalizeTokenUsfm (ParatextData/UsfmToken.cs) emits a line break before every Paragraph token, so P9 gives you the empty \p and then the pasted paragraph. A paste now inserts what was pasted and nothing else, and if that means two paragraph markers in a row, that is what the bytes say — the same shape typing them produces.

Removing it takes the whole paste-provenance apparatus with it: the isPasteRebuild parameter through $buildParaFragment/$rebuildParas/$requestTier2ForNode, Tier2Context.pasteRebuildArmed, the pastePendedKeys set that carried "this rebuild came from a paste" across a deferred settle, its per-commit pruning, and consume-on-use. The read-only settle and the mutating rebuild now call $buildParaFragment with identical arguments, which also closes a review finding: the two could disagree about a deferred paste's shape, letting EditorRef.getUsj() predict a paragraph the settle would not produce.

It also exposed a real defect the dedup had been masking. A multi-line paste replays each line break as a paragraph split, and the engine injects a marker prefix onto every fresh split paragraph — including onto lines that already carry their own marker. For a marker with a separator and content the injected prefix resolves away; for \b, which has neither, both survived, so every blank line in a pasted document gained an empty \p in front of it — 16 of them in the 2sa fixture. $paraMarkerDeletionTransform's split branch now skips the injection when the paragraph's own text already opens with a paragraph-marker literal. Enter pressed directly in front of a marker literal is the same shape and gets the same answer.

The round-trip suites seeded that empty \p host and expected the paste to swallow it; they now assert it is there and strip it, so a genuine stray paragraph can no longer pass as the harness's own.

Correctness fixes (24b030d2)

  • A foreign Lexical editor's clipboard was treated as internal. isInternal answered the mere presence of application/x-lexical-editor, but Lexical's fast path is namespace-scoped ($insertDataTransferForRichText requires payload.namespace === editor._config.namespace). A payload from the app's own "Commenting" editor (CommentPlugin.tsx) made every claim stand aside, then fell through to the text/html DOM import — which runs neither the \c/\id strip nor the NBSP rule, so a \c copied out of a comment box reached the document unstripped. getPastePayload now mirrors Lexical's actual condition. (A sibling test already called this the "same-namespace-flavor guard", so the code was behind its own documented intent.)
  • The char-stack line replay ignored structure protection. Registered after the Standard-view claim at the same priority and before StructureKeyboardPlugin mounts, so the one paste that handler declines reached it and was replayed with removeText() plus a split per line. Measured with the guard removed: a protected paste deletes \v 18 and the \nd span and splits the paragraph — exactly what protection had just refused.
  • LEADING_MARKER_LITERAL accepted a closer (inherited from the shared terminated-marker shape), so a leading \zbold* read as the paragraph marker zbold — unknown markers are paragraph-kind — and the host's glyph was dropped: an \s1 heading became \p. This matched P9's own rule exactly once fixed: // End tokens are always end tokens, even if unknown (UsfmToken.cs). (Moot now that the dedup is gone, but the classification fix stands.)
  • A copy that produced no bytes still overwrote the clipboard with an empty string and an empty <p> — the same loss the collapsed-selection guard exists to prevent, reached through the non-collapsed door. The event is claimed and nothing is written.
  • A CR inside a decoded Paratext 9 note survived usfmFromComment and became a hard line break, splitting the note across two paragraphs.
  • DecoratorBoundarySelectionPlugin pointer tracking: the press was heard on the root while the release was on the document, so a press landing just outside the contenteditable root started a drag the plugin never saw; and a secondary-button press could leave the flag stuck when the native context menu swallowed the pointerup. Both fixed.

\periph keeps its division when the attribute list is refused (a0efe7df)

\periph has no closing marker, so a peripheral division's extent is a property of the marker, not of its attributes. Refusing the marker line's attribute list was demoting the whole division to an ordinary paragraph and re-parenting every block under it to the root. P9 reads it the other way: a failed SetAttributes leaves the marker-line text alone and the USX writer still opens <periph> with an empty id. The refused bytes now survive as the division's own title and serialize back byte-for-byte. This also removes a live-typing glitch — typing |id="cover" onto a periph line used to un-nest everything under it on the | keystroke.

Splitting the marker line at its FIRST | is unchanged: that matches P9's own text.IndexOf('|').

Test defects found and fixed

  • The optbreak carrier-agreement pin asserted not.toContain against a flavor that is always "" for that selection (the opaque-block rule suppresses it), so reverting UnknownNode.isSelected left it green. The override's range branch — its whole reason for existing — had no pin anywhere; two now live in UnknownNode.test.ts, verified to fail when the override is reverted.
  • A mixed-selection pin asserted only that the document survived, never that the paste landed.
  • The accepted P9 leading-space parity was parked in KNOWN_LOSSY, hence skipped, hence protecting nothing despite the file calling it "NOT a fix candidate". It is now pinned positively.
  • Two internal-paste pins passed "{}" as the lexical flavor — never same-namespace — so they pinned the old presence-only rule.
  • Two pre-existing possibly undefined type errors in the protected-paste suite (present before this round) are fixed.

Every behaviour fix above was verified by reverting it and watching the new test fail.

Verification

shared 700, shared-react 1670 (+1 skipped), platform-editor 1914 (+3 skipped); per-package typecheck, eslint and prettier clean; verify-committed-dist ✓.

Review round (lyonsil, 2026-09-18)

Fixed in 0c701229, b17f235c, de737c0a, d478270c, 056d9439, 17018ce0 (dist 4b91f3ae); every behaviour fix has a test that fails without it.

Paste

Copy / cut

Positions

  • USJ positions after a milestone, a verse's \va/\vp or a chapter's \ca in Standard view were one content item too far — the attribute display run's wrapper was counted as content (056d9439). This affected annotations as well as the new Markers-view copy; pinned in logical-content-agreement.test.tsx. The same skip is part of PT-4370's #11 (acff4371), which also stops note callers and editable chapter content from counting as content items; this PR keeps its copy because the Markers-view copy needs it. See "Merging alongside PT-4370: Settled positions across the public position API #11" below.

Docs / cleanup: docs/clipboard-semantics.md records each rule and the known limits (a data NBSP inside a Paratext 9 note's usfm: comment #15; hidden-marker views' text/html #23; the text/html blank-line limit #31). Spec-section labels are gone from code and tests (#26, #30), and the pointer-release editor.update() exception is justified at the site (#21).

Deferred to PT-4643: #3 (a drag from a figure glyph can die right after a caret host is evicted) and #4 (the cut-through internal-flavor rule in Formatted/Paragraph-structure views). Details and repros are on the ticket.

Not changed (reasons on each thread): #7 (not reproduced), #10 (measured in Electron: a copy event cancelled with nothing written leaves the clipboard untouched), #22, #25.

Tracked elsewhere: a lone data NBSP dropped on save in the non-Standard views is PT-4391.

Verification

platform-editor 1959 (+3 skipped), shared 701, shared-react 1671 (+1 skipped); typecheck, eslint and prettier clean; verify-committed-dist ✓. The companion e2e (paranext-core #2656) passed against a build of this branch before the last four commits (protected-cut arm, NBSP before a marker, the attribute-run position fix, the Markers-view copy). None touches a path that spec drives: it runs in Standard view, where MarkersViewCopyPlugin is not mounted, on Jonah, which has no milestone, \va/\vp or \ca. The Markers-view copy has not been tried in the app yet.

Review round 2026-09-22 (22 findings)

17 fixed, 5 not taken; a reply on every thread and a summary comment covering the three that could not be attached to a line. Every behavioural fix has a test that was made to fail against the unfixed code first.

Structure protection was being pre-empted by handlers that outrank it (d4eb5d3c)

  • The in-note multi-line paste claim runs at CRITICAL and removes the selected range before deciding anything, so under structureProtectionMode: "protected" a paste over a selection reaching out of a footnote across a verse marker destroyed the whole range before the refusal ran. Its test reduces the document to the pasted text without the guard.
  • The paragraph-reap arms were set from the pre-gesture selection with no such check. A refused keystroke or cut mutates nothing, so nothing commits and the update listener that resets the arms never runs; the arm survived into the next unrelated commit, where a paragraph transiently emptied by a rebuild read as that gesture's provenance and was reaped. Neither gesture arms now if protection will refuse it. That turned out to be only one of the refusals involved; see the follow-up below.
  • StructureKeyboardPlugin's CUT refusal moved to COMMAND_PRIORITY_CRITICAL (KEY_DOWN stays at HIGH): a refusal has to outrank the actor it refuses rather than tie with it on mount order. This also means structureProtectionMode: "protected" is honoured in the Markers view whatever the plugin order.

A range framed by a span's own read-only glyphs copied nothing (187f24df). An element selection point on a marker glyph resolved to a location keyed only on the parent node, so "before the opening glyph" and "before the closing glyph" were the same location; a range framing a char span or note rebuilt collapsed and the Markers-view copy claimed the event while writing nothing — a copy that looked like it worked. $getLocationFromNode now emits a UsjClosingMarkerLocation for the closing glyph. Reachable by a click on one glyph followed by a shift-click on the other (not by a single drag). Pre-existing in selection.utils.ts; this PR's Markers-view copy is the first caller to surface it.

A cut in a read-only editor removed text (f56ece0c). ClipboardPlugin's Ctrl+X listener is bound to the root element in every view and dispatches CUT_COMMAND with no editability check, so a read-only Standard view reached selection.removeText(). $writeCopyPayload now owns the editor.isEditable() rule for both its callers.

A pasted space ahead of a \c/\id token was dropped (f56ece0c), along with its line break — the widened blank-remainder check took whitespace the user wrote as if it were the token's own separator. The line now goes only when the token OPENS it.

Other fixes: formatPara and insertNote commit discretely like every other content-mutating EditorRef method (2ac599b1); Editor warns when markerMode: "visible" is mounted without isReadonly, since no marker repair runs there; the in-note claim gates on a raw newline before its strip/normalize pass; the two marker-adjacent NBSP passes merged into one alternation (9dd93da5, equivalence checked over 5,229,057 inputs); plus the Paratext 9 signature constants, plugin ordering, getAllTextNodes(), an ancestor-walk helper, a typeof guard and test-tree shape.

Not taken (reasons on each thread, which are left unresolved): an NBSP between \va* and \vp becoming a space and unfolding \vp — that is what Paratext 9 produces from the same bytes with AllowInvisibleChars off, and dropping the byte would lose content the user pasted, so it is pinned as a test instead; the Markers-view copy's per-copy chapter rebuild (bounded by one chapter, its own piece of work); a NodeSelection over a read-only construct (no demonstrated path); and structural sanitization of the internal lexical-JSON paste payload (unreachable today — the copy side already refuses book/chapter — and wants its own tests).

Follow-up 2026-09-22: other refusals that left a paragraph-reap arm behind

Two findings on the fix above (dd576918). Structure protection is not the only handler that refuses a gesture after the engine has armed. The opaque-block guard refuses a Delete or a Cut whose selection ends inside a figure caption, in every protection mode. Guarded mode's first two-step press over a verse marker changes React state only. Neither commits, so the arm survived and a later transient emptying reaped the paragraph. Each case has a test that fails without the fix.

  • Delete keys: the arms now also expire on a microtask. This is safe because Lexical's rich-text delete keys run to completion inside the same KEY_DOWN dispatch, and the paragraph transform consumes the arm in that update. It also covers any refusal added later.
  • Cut: asks $selectionReachesIntoOpaqueBlock() before arming. It does not use the microtask. Lexical's own cut, which Unformatted view uses because it has no Standard-view claim, removes the range only after await copyToClipboard(...). A microtask expiry would drop the arm before that removal and leave a whole-cut paragraph behind as an empty husk. A new test pins this, and it fails with the expiry applied to the shared arm functions.

Also fixes CI: four spec-file TS18047 errors from the round above (requireDefined narrows undefined, not null), in 2c762a47.

Merging alongside #11 (PT-4370)

This PR and #11 can merge in either order. 056d9439's AttributeRunNode skip is also in #11's acff4371, which is the complete version. Whichever PR merges second resolves three conflicts:

A trial merge resolved this way passed all three suites on 2026-09-22 (shared 728, shared-react 1757 +1 skipped, platform 2091 +3 skipped), including this PR's Markers-view copy tests and the milestone pin.


This change is Reviewable

@lyonsil lyonsil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of the copy/paste USFM fidelity work on this branch — the Standard-view clipboard path, the
paste normalization passes, the Paratext 9 decoder, the marker-edit engine's paragraph handling, and
the new decorator-boundary selection plugin. 25 findings, most of them in
whitespaceDisplay.plugin.utils.ts and MarkerEditPlugin.tsx.

Each comment leads with a severity and a short status. The severity is the one settled on after the
finding was checked against the code, not a first impression. checked and confirmed means the
behaviour was reproduced or traced to its cause in this branch — where a comment says something was
run, it was run. Every comment ends with a concrete Fix:; where a fix could not be stated as a
verified edit, it is stated instead as the constraint any fix has to satisfy, and several name an
accompanying test, doc, or changelog update the change would need.

Findings that could not be attached to a line in this diff

#4 (high · checked and confirmed) libs/shared/src/nodes/features/UnknownNode.ts:253 - In Formatted and Paragraph-structure views, copying part of a figure's caption puts the entire figure on the clipboard.

What happens: the over-carry is mitigated only inside $getStandardViewClipboardData (whitespaceDisplay.plugin.utils.ts:836, the $selectionReachesIntoOpaqueBlock omission), and $handleCopyForStandardView is registered only when isStandardView (MarkerEditPlugin.tsx:872). In hidden-marker views Lexical's own copy runs, so $appendNodesToJSON decides everything: the new excludeFromCopy at :253 answers false for any child-bearing construct and the new isSelected at :288 answers true when the caption child is in getNodes(). createUnknown stamps mode: "token" on every text child and $sliceSelectedTextNodeContent refuses to slice those, so the whole construct is serialized.

Why it matters: the hazard the CHANGELOG describes for Standard view — a native paste inserts a second figure and a save persists it — is now reachable in the other editable views.

Fix: do not implement this by changing UnknownNode.excludeFromCopy/isSelected — Lexical hoists an excluded wrapper's children, and the caption child is mode: "token" and unsliceable, so that would emit the caption's full text as loose prose with no wrapper or attributes. Instead add a view-agnostic guard beside $selectionReachesIntoOpaqueBlock in libs/shared-react/src/plugins/usj/OpaqueBlockGuardPlugin.tsx (or wired into libs/shared-react/src/plugins/usj/clipboard.utils.ts) that runs on COPY_COMMAND/CUT_COMMAND in every editable view and, when $selectionReachesIntoOpaqueBlock() is true, keeps application/x-lexical-editor off the clipboard for that copy — mirroring $getStandardViewClipboardData — leaving Lexical's text/plain as the fallback. Pin a Formatted-view partial-caption copy in unknownClipboardFidelity.test.tsx, update docs/clipboard-semantics.md and packages/platform/CHANGELOG.md since this changes clipboard behaviour outside Standard view, and rebuild the committed packages/platform/dist.

How this was checked: The premise holds. MarkerEditPlugin.tsx:564 (if (!isEnabled || !viewOptions) return;) early-returns the whole registration effect for Formatted and Paragraph-structure views, both markerMode: "hidden" (view-options.utils.ts:212-236), so the COPY/CUT handlers at :872-895 never mount there, and OpaqueBlockGuardPlugin explicitly leaves copying untouched. Lexical's stock COPY handler then runs $appendNodesToJSON, which calls excludeFromCopy('html') — never 'clone' — returning false for a child-bearing figure at UnknownNode.ts:253-254, while isSelected at :288-298 returns true because the partially selected caption TextNode is in getNodes(). $sliceSelectedTextNodeContent refuses to slice a token-mode TextNode (LexicalSelection.dev.mjs:257), and createUnknown (usj-editor.adaptor.ts:850-852) stamps mode: "token" on every child regardless of markerMode. So the whole figure, with the caption's full untouched text, serializes into application/x-lexical-editor from a selection covering only part of the caption.

Not attached to a line: no longer on a changed line; cannot be posted inline

#5 (high · checked and confirmed) packages/platform/src/editor/markerEdit/MarkerEditPlugin.tsx:365 - Pasting a \c-bearing multi-line payload into an expanded note deletes the note's selected content and inserts nothing.

What happens: :365 tests the UNSTRIPPED payload.text for a newline, and only then does :378 compute normalizePastedNbsp(stripPastedChapterAndBookId(pastedText)) — and the strip drops whole lines (whitespaceDisplay.plugin.utils.ts:344). For "\\c 5\n\\id MAT" both lines go, leaving lines = [""]. $handlePasteLinesInNote (markerEditNote.utils.ts:113) removes the selection via $removeSelectionTouchingExpandedNote, inserts "", and returns "handled", so :390 calls preventDefault() and the CRITICAL claim consumes the command.

Why it matters: content is deleted with nothing pasted. The single-line variant is wrong too: "\\c 5\nfootnote" collapses to one line but still takes the \fp path and its stripLeadingParagraphMarker, which the genuine single-line path never applies.

Fix: decide emptiness before any selection mutation — $removeSelectionTouchingExpandedNote's selection.removeText() runs as a side effect of merely checking whether the paste applies, so declining afterwards cannot undo it. In the in-note CRITICAL claim, when isStandardView, compute const noteText = pastedText.includes("\n") ? normalizePastedNbsp(stripPastedChapterAndBookId(pastedText)) : pastedText; up front; if it is empty, call event?.preventDefault() and return true before calling $handlePasteLinesInNote, leaving the selection untouched. Do not copy the order at $handlePasteForStandardView:576 — it carries the same defect. Re-run the newline test on noteText rather than pastedText, so "\c 5\nfootnote" collapses to the ordinary single-line path instead of $handlePasteLinesInNote's stripLeadingParagraphMarker. Pin both shapes in noteEnterFp.test.tsx. Rebuild the committed packages/platform/dist (node scripts/rebuild-committed-output.mjs).

How this was checked: Confirmed empirically. In the in-note CRITICAL PASTE_COMMAND claim, the newline test at MarkerEditPlugin.tsx:365 runs on the unstripped pastedText; the strip happens only afterward at :378-379. For "\c 5\n\id MAT", stripPastedChapterAndBookId drops both lines, so lines is [""]. $handlePasteLinesInNote (markerEditNote.utils.ts:113) first calls $removeSelectionTouchingExpandedNote, which unconditionally does selection.removeText() before it knows the pasted content is empty; $insertLineTextAtCaret("") is then a no-op and the claim returns "handled", so preventDefault() fires at :395 and the command is consumed. Verified with the renderStandardEditorWithUnclosedNote fixture: selecting "no" out of a note's "A note" content and pasting "\c 5\n\id MAT" left the note as "A te" — the selection deleted, nothing inserted. The single-line variant also diverges as described: "\c 5\nfootnote" still enters the multi-line branch and gets stripLeadingParagraphMarker, which a genuine single-line paste of "footnote" never receives.

Not attached to a line: no longer on a changed line; cannot be posted inline

#10 (high · checked and confirmed) packages/platform/src/editor/markerEdit/whitespaceDisplay.plugin.utils.ts:902 - A copy with nothing to put on the clipboard still wipes the clipboard when it arrives as a real browser copy event.

What happens: the null-dispatch branch at :893 skips copyToClipboard entirely for an empty payload, so the clipboard is untouched. The native-event branch does not: :902 cancels the browser's own copy unconditionally and :903 then writes nothing. A canceled copy event still commits its DataTransfer, and Blink's SystemClipboard::WriteDataObject replaces the whole clipboard — so an empty object clears it.

Why it matters: the comment at :880 promises "leaving whatever the user copied last intact". It is reachable from any host path that delivers a real ClipboardEvent — an Electron Edit-menu Copy, or a host that does not route through ClipboardPlugin's keydown.

Fix: in $handleCopyForStandardView, restructure the native-event leg so event.preventDefault() and the setData loop both sit inside the !isEmptyPayload branch, leaving preventDefault() uncalled when the payload is empty so the browser's own no-op-on-nothing-to-copy default runs — mirroring the collapsed-caret branch at :872-877, which already declines without calling it. Pin both dispatch legs in the empty-copy describe of whitespaceDisplay.plugin.utils.test.tsx: the null-event leg, and a real ClipboardEvent-shaped stub with a spyable preventDefault, asserting it is NOT called on the native leg when the payload is empty. Add a line to the empty-copy rule in docs/clipboard-semantics.md covering a real native ClipboardEvent reaching a non-collapsed, zero-byte selection, not only the collapsed-caret case already documented. Rebuild the committed packages/platform/dist (node scripts/rebuild-committed-output.mjs).

How this was checked: Confirmed the asymmetry directly: git diff against the merge base shows this PR added the isEmptyPayload guard around the null-dispatch leg's copyToClipboard call (:893) and around the native-event leg's setData loop (:903-904), but left the native leg's event.preventDefault() at :902 unconditional, exactly as it was pre-PR. The native-event leg is genuinely reachable: Lexical's core attaches a real copy DOM listener to the editor root and dispatches COPY_COMMAND with the actual ClipboardEvent, and MarkerEditPlugin.tsx:879-891 forwards it unchanged into $handleCopyForStandardView at COMMAND_PRIORITY_HIGH, above Lexical's own fallback — so any host path delivering a genuine native copy event, an Electron or OS Edit-menu Copy being the standard example, reaches this branch. Within this repo's own UI, ClipboardPlugin and ContextMenuPlugin only dispatch via copySelection/cutSelection, which pass null (clipboard.utils.ts:60,66). The sibling collapsed-caret branch a few lines above (:872-877) establishes the intended precedent: for a real ClipboardEvent with nothing to copy it returns false without calling preventDefault, precisely so the browser writes nothing — docs/clipboard-semantics.md:27 states this outright. The empty-payload branch breaks that pattern.

Not attached to a line: no longer on a changed line; cannot be posted inline

#27 (low · checked and confirmed) packages/platform/src/editor/markerEdit/whitespaceDisplay.plugin.utils.ts:199 - One Ctrl+V parses the pasted HTML six times before a byte is inserted.

What happens: getPastePayload unconditionally computes htmlText at :201 (a DOMParser.parseFromString plus three querySelectorAll sweeps) and paratext9Text at :221 (a second parse, a comment TreeWalker, and a querySelectorAll("*") over every element) before isInternal or the selection is consulted. Three handlers each call it for the same event: $handlePasteForStandardView (:548), the in-note CRITICAL claim (MarkerEditPlugin.tsx:362) and the char-stack claim (MarkerEditPlugin.tsx:433). htmlText is only ever read by plainText || htmlText, which essentially no real clipboard reaches.

Why it matters: pasting a Word, Outlook or browser payload — hundreds of KB is routine — blocks the main thread inside editor.update() for six full parses. The base commit did one.

Fix: gate paratext9HtmlToUsfm behind html.includes("usfm:") || html.includes("usfmopen") || html.includes("usfmclosed") — every branch of hasParatext9Signature requires one of those literal byte runs, so this cannot silently disable the P9 decode. Make the local htmlText in getPastePayload (:201) compute on demand inside the plainText || htmlText fallback; it is not a PastePayload field, so no consumer is affected. For the repeat calls, prefer computing the payload once in the shared caller over a module-level cache: Lexical's triggerCommandListeners hands every priority tier the same payload reference for one dispatchCommand, so no cache is needed. If a WeakMap is used anyway, guard the key — read and write it only when event !== null && typeof event === "object", because WeakMap throws on a null key and this function's signature admits null. Add a call-count assertion (spy on DOMParser.prototype.parseFromString) in whitespaceDisplay.plugin.utils.test.tsx proving the protected + char-stack path parses once per event, and rebuild the committed packages/platform/dist.

Not attached to a line: no longer on a changed line; cannot be posted inline

(AI-assisted, with my guidance)

@tjcouch-sil

tjcouch-sil commented Sep 21, 2026 •

Copy link
Copy Markdown
Member Author

Thanks for the thorough review. Every inline finding has a reply on its thread; the four that could not be attached to a line:

  • Consider committing built dist/ like platform-bible-react, so consumers stop building at install time #4 (Formatted/Paragraph-structure partial-caption copy writes a whole figure into the internal flavor) — folded into PT-4643: it is the same cut-through rule Standard view now applies, extended to the views that run Lexical's stock copy. The symptom, cause, your fix shape (a view-agnostic COPY/CUT guard, not excludeFromCopy/isSelected) and the pins are recorded there.
  • Separate the published artifact directory from the toolchain's scratch space (both are currently dist/) #5 (in-note paste of a \c-only multi-line payload deleted the note's selected text) — fixed in b17f235: the in-note claim now strips first and tests for a line break on the stripped text, so an empty or single-line result is left to the Standard-view claim, which keeps the selection for an empty one. Pinned in noteEnterFp.test.tsx.
  • PT-4313: Compensate every indented paragraph marker in the gutter view #10 (native copy event with an empty payload clears the clipboard) — not changed, because the premise does not hold in our Chromium. Measured in the app's own Electron: with a sentinel on the clipboard, a copy event that calls preventDefault() and writes nothing left the sentinel in place, for both a text selection and a byte-less range (matching the Clipboard API spec, which writes only a non-empty item list). So the native leg already writes nothing; dropping preventDefault would only hand the copy to the browser's default action.
  • PT-4703: Ask before an edit removes content the view does not show #27 (six html parses per paste) — fixed in b17f235: the P9 decoder checks its signature bytes (usfm:, usfmopen, usfmclosed) before parsing, and the generic html decode is lazy, so a paste whose text/plain wins and whose html is not P9's parses nothing (pinned with a DOMParser.prototype.parseFromString spy). I did not add per-event memoization: with both gates, the three claims no longer parse anything in the common case.

The PR description has a summary of the whole round. Pushed: 0c701229, b17f235c, de737c0a, d478270c, dist 50902bf4.

(AI-assisted)

tjcouch-sil and others added 23 commits September 22, 2026 11:59
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A copy that does not come from a real browser clipboard event — Ctrl+C/Ctrl+X,
the context menu's Cut/Copy, EditorRef.copy()/.cut() — dispatches the command
with a null payload, and @lexical/clipboard has to synthesize an event before
anything can be written: it appends a hidden placeholder element to the editor,
points the DOM selection at it, and runs document.execCommand("copy"). Filling
that synthesized event declines an empty selection BEFORE suppressing the
browser's own copy, so the browser copies the placeholder and the user's real
clipboard contents are replaced by a character that was never in the document.

A read-only construct makes it likelier, not special: an UnknownNode renders
contentEditable=false and its marker glyphs are decorators that take no keyboard
selection, so trying to select a figure's marker text leaves the caret in the
prose beside it. A plain collapsed caret does the same, in every view, and cut
takes the identical path.

Two layers. copySelection/cutSelection (shared-react) do not synthesize a copy
for a selection that cannot produce content, and leave the key unclaimed so the
browser's own no-op copy runs — that is what "untouched" means. And the
Standard-view handler claims a null-payload dispatch it cannot build data for
rather than declining it into @lexical/rich-text's fallback. A real clipboard
event at a collapsed caret is still declined, and a node selection still goes to
Lexical, which copies it correctly.

Copying a read-only construct works through a selection that contains it — the
shape a browser resolves a drag over one into — which reaches its display glyphs
through the same walker as prose and yields the construct's real USFM bytes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…arker kinds

The paste carrier choice now states its trade-off where it is made. Choosing by
which carrier an NBSP survived in is the better rule for a foreign clipboard,
and unusable here: Standard view's own text/plain deliberately carries no NBSP
at all, so an NBSP-presence test is true of every P10 copy and would route the
editor's own round trip through html, whose decoded text drops a collapsed
note's caller. Recorded in the semantics doc's deferred list too.

PastePayload keeps only what a claim may act on. The unresolved carriers were
exposed and unread; every claim must replay the same bytes, and a second carrier
within reach invites one of them to pick differently.

Doc corrections: getPastePayload named three PASTE_COMMAND handlers and
described the Standard-view claim as NBSP-gated, both false since the merge;
isInternal claimed that claim declines internal payloads outright, false since
the attribute path claims them; $handlePasteForStandardView lost its
"call inside editor.update()" contract note while gaining a $getEditor() call
and a command dispatch that make it matter more.

The marker-kind classifiers move to a leaf module. Both tiers ask the question
and must answer it identically, and Tier 2 reaching into Tier 1 for it made the
two import each other. Their only inputs are a string and a stylesheet lookup,
so neither tier is needed to answer it.

Also restores the multi-paragraph-html paste test's return-contract assertion,
lost when it switched to a command dispatch — the content assertion alone cannot
tell this handler's replay from Lexical's own html import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rsion

The copy flavors no longer split into "text/plain inverts, everything else
ships the display form". text/html inverts too, collapse-awarely: an interior
single display-NBSP becomes the plain space it stands for, while a run of 2+ and
a fragment edge keep NBSP because a rich-text consumer collapses runs and drops
edge whitespace. Only the internal Lexical flavor is still untouched, so an
internal paste round-trips the display form exactly.

That also sharpens why the paste carrier is chosen by presence rather than by
where an NBSP survived: Standard view's own text/plain still carries no NBSP at
all, while its text/html now carries them precisely where a plain space would be
destroyed — so an NBSP-presence test would route every P10 round trip through
html and lose the note caller.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Checking the selection before dispatching read the COMMITTED editor state,
which Lexical is a microtask behind — so a selection made earlier in the same
tick was invisible and the copy was silently withheld. That is the ordinary
shape of the public EditorRef.copy() path: select something programmatically,
then copy it. Reading the live state was not a drop-in fix either, since the
context menu invokes its options from inside editor.update().

The guard is now a COPY_COMMAND/CUT_COMMAND listener registered above
@lexical/rich-text's fallback and below every feature handler. A command
listener runs inside the update, where the selection is authoritative for
pending and committed state alike, so the staleness class is gone rather than
narrowed — and one registration covers the shortcuts, the context menu and the
editor ref, since all of them dispatch the same command.

The three Standard-view pins asserted nothing that could fail. The dispatch's
return value is true either way (rich-text always claims), the clipboard-module
spy cannot see a call made through the externalized rich-text import, and the
execCommand assertion was masked by @lexical/clipboard's module-level latency
timer left set by the preceding test. They now watch for the dispatch reaching
past the claim — immune to module state and test order — and drain that timer,
and all three fail in a whole-file run when the claim is reverted.

Also pins the node-selection carve-out (a copy that has content must still
happen), the context-menu and editor-ref legs, records the empty-selection
no-op on the public API's TSDoc, and marks the browser's drag-over-a-read-only-
block selection shape as the assumption jsdom cannot verify.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…idened MarkerEditContext

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
Pasting `\fig At once they left their nets.|src="avnt016.jpg" size="span"
ref="1.18"\fig*` into Standard view left a figure box holding only
`\fig |src="…"\fig*`, the caption stranded after the closing glyph as
paragraph prose, and an exported USJ whose `figure` object had no content
at all — the caption merged into the neighbouring text run.

`$textNodeInUnknownTransform` (TextSpacingPlugin) ejects any TextNode that
is new in the current update out of an UnknownNode parent, so a user typing
into a read-only opaque block never plants prose inside it. A Tier-2 rebuild
materializes the whole figure — wrapper and caption together — in one update,
so the caption looked exactly like typed text to that rule. Eject only when
the wrapper predates the update, which is the shape a typed intrusion
actually has.

The class is invisible to a MarkerEditPlugin-only harness (the ejection lives
in a sibling plugin), so the new pins mount the full Editor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
…copy

An internal (`application/x-lexical-editor`) copy→paste of a figure, sidebar,
periph or ref pasted the construct's full literal USFM bytes onto the screen
while the exported USJ silently dropped the node and every attribute on it —
a convincing display over lost data that a save then persisted with no error.

`UnknownNode.excludeFromCopy` kept only a child-bearing optbreak out of the
exclusion; every other kind stayed excluded, and `$appendNodesToJSON` does not
drop an excluded node — it hoists its children into the parent, stranding the
content-free display decorators that render each kind's marker and attribute
bytes. Key both `excludeFromCopy` and the `isSelected` boundary override on the
node's own child count rather than on its kind: the kinds share one
display-decorator shape, so they shared one failure.

Verified per kind across all three real clipboard payload shapes (plain-only,
plain+html, and the full payload carrying the lexical flavor). Tables are not
part of this — `table`/`table:row`/`table:cell` are their own `ImmutableTable*`
nodes now and never took the exclusion — and are pinned in the same sweep to
say so. The plain-carrier losses each kind genuinely has (a block construct
split across lines; a `ref` wrapper USFM has no bytes for) are asserted as
losses rather than left unpinned.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
A sidebar's nested `\p` and a table's rows and cells are real block-level
nodes, so the copy walker put a `\n` at each of them and spread ONE construct
over several lines. Paste replays every `\n` as a paragraph split and Tier 2
re-tokenizes strictly one paragraph at a time, so the `\esb`/`\esbe` and
`\tr`/`\th`/`\tc` assembly the fragment tokenizer already implements never saw
the whole construct: a pasted sidebar came back unclosed with its paragraph
hoisted out, a pasted table as empty row wrappers with the cells stranded as
sibling paragraphs.

Suppress the line break for a block boundary whose opaque ancestor is above it,
so the construct stays byte-contiguous and a single-paragraph rebuild can
reassemble it. The construct's own outermost node still starts its own line, so
a table stays separated from the prose around it. USFM markers self-delimit, so
the one-line form is valid USFM of the same document — just not the
line-per-marker layout a writer emits.

Tables needed one more thing on the paste side: usfm.sty classifies
`\th1`/`\tc1`/… as CHARACTER styles, so with a real project stylesheet a cell
marker reached the tokenizer as a char token and the table-cell assembly, which
only ran for paragraph tokens, never saw it. ParatextData derives a cell from
the marker NAME, so recognize one on either token kind, guarded on an open row.

The corpus sweep's sidebar and table fixtures are un-skipped and now sweep
clean. `\periph` is not fixed: the tokenizer has no periph assembly and periph
renders attributes as a pipe run on a marker with no closing bytes. That is
recorded and pinned as a loss rather than left to look fixed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
A paste's own update does not always get to perform the rebuild its bytes ask
for. A paragraph whose re-tokenization would EJECT content out of a milestone
deliberately waits for caret departure, and by then `pasteRebuildArmed` — a
per-commit flag — has reset, so the departure settle rebuilt the pasted line as
though the user had TYPED it: the host's `\p ` glyph and the pasted line's own
`\p ` literal both survived, leaving a stray empty paragraph ahead of the
correct one.

`Tier2Context.pastePendedKeys` gives the arm a lifetime tied to the pended work
instead of to the commit clock. A pend recorded while the paste's own update is
armed records its key; the settle that routes that key to a rebuild consumes the
provenance and passes it down. Keying it that way is what keeps it out of typed
input: keys are only added inside a paste's own update, consumed by the first
rebuild that reaches them, and pruned by the per-commit update listener as soon
as they stop pending — and text typed beside the pasted bytes pends its own key,
which carries none.

Closes the last "Known Lossy" clipboard entry. The corpus `milestones (ts)`
fixture it was filed against does not in fact reach the deferred path (its
milestone ejects nothing), so it is un-skipped and swept clean; the shape that
does reach it is pinned directly, together with its counter-pin that the same
bytes with no paste behind them still split with an empty predecessor.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
Four review follow-ups on the opaque-construct clipboard cluster.

`UnknownNode.excludeFromCopy`'s rationale still read as an optbreak husk rule
after the guard was generalized to child count. It now names what the rule
actually excludes, including the case it was under-describing: a hidden-marker
view builds no display children at all, so ANY content-less construct there — a
caption-less figure, an empty ref — is childless and dropped. Recorded as a
deferred item rather than changed, since a marker-free view copying
marker-only constructs is a view-semantics decision.

Pinning the `isSelected` boundary on a second kind turned up a real
disagreement rather than confirming the shared rule. A construct whose display
bytes lead starts with a decorator, so an element-type boundary point on the
wrapper stays one and reaches no child; a construct with no display bytes (a
`ref`) starts with a real TextNode, and Lexical normalizes that same point into
a text point at the child's offset 0 — the child is then in `getNodes()` while
contributing zero characters, so the wrapper and its whole content went into
the lexical flavor while `text/plain` had none of it. `isSelected` now asks
whether the selection covers a child's CONTENT, which is what the copy walker
computes. Pinned for figure and ref in both directions.

The tokenizer's character-token cell arm gains sheet-backed unit coverage for
the align-infix names and both of its guards; a ranged cell is not a sheet
entry at all and still arrives as a paragraph token, so that is pinned too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
…ts text

The content-coverage `isSelected` predicate closed the `ref` boundary's carrier
disagreement by trading it for something worse. Excluding the wrapper does not
drop it quietly: `$appendNodesToJSON` hoists its children in its place, and
`createUnknown` stamps `mode:"token"` on every text child, which
`$sliceSelectedTextNodeContent` refuses to slice — so the zero-width child kept
its full text and the copy carried `Genesis 1:1` with the `ref` node and its
`loc` attribute silently gone. That is the convincing-lie shape the suite exists
to prevent, and the tag-only assertion could not see it.

Revert to child membership, which keeps the construct whole and makes the
lexical flavor a superset of `text/plain` at that one boundary. Making it appear
in neither carrier is not reachable from the node API: the children are plain
TextNodes, and `exportNodeToJSON` requires every ElementNode's `exportJSON()` to
return a `children` array, so a node cannot exclude itself AND its children. The
residual is recorded in the deferred list with both measured payloads.

Every boundary assertion now reads the construct's own content bytes, and the
pins state the behavior per kind — a decorator-led construct is excluded from
both carriers, a text-led one is carried whole — so the lossy variant turns them
red instead of passing.

Also scoped the provenance doc to what was measured: typing into a DIFFERENT
node pends its own key and takes the typed path, but typing into the still-
pending node itself adds no key, so the paste path runs and the dedup applies to
the line those bytes are now part of.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
A copied `\periph` came back as an ordinary paragraph carrying its own attribute
bytes as literal content: `usfmFragmentToUsj.ts` had assemblies for `\esb`/`\esbe`
and `\tr`/`\th`/`\tc` but none for `\periph`.

The suspected second cause — that the one-line opaque-construct copy removes the
attribute run's only delimiter — does not hold. Measured: the one-line and
line-per-marker spellings tokenize identically, because a line break is ordinary
whitespace and the next `\marker` is what ends a division's marker line either way.
The one-lining is in fact required: a paste replays every line break as a paragraph
split and Tier 2 re-tokenizes one paragraph at a time, so only the one-line form
lets a single pass see the division together with the blocks inside it. Pasting the
line-per-marker spelling was measured to come back as an empty division with its
paragraph beside it, and that measurement is now a pin.

`\periph` therefore assembles like a sidebar: the marker line splits at its first
`|` into the division title (`alt`) and a named-attribute list, and the division
takes every following block until the next `\periph`, a chapter, or the fragment
end. A division has no closing bytes, so there is no unterminated state to record.
An attribute list that does not parse degrades the whole division to a literal
paragraph, as an unfoldable figure span does.

A sidebar now lands in an open division instead of beside it, which required
pushing it before assigning `sidebar` — `blockTarget()` answers "the open sidebar"
whenever one is open, so assigning first nested it inside itself.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
Deferred item 9's two attribute-context paste residuals, characterized and closed.

(a) does not reproduce and is not paste-specific. A mixed selection reaching past a
char span's closing glyph into following sibling text settles to byte-identical USJ
whether the replacement is typed or pasted, across four selection shapes (forward
and backward, run-start-to-trailing-end and run-end-to-trailing-start included). The
span coming back `closed="false"` with its attribute run collapsed into content is
the arithmetic of a range that covered the closer, not a defect: deleting a range
containing the closing glyph deletes the closing glyph.

(b) reproduces, from BOTH directions, and is a data-fidelity hole. The value-byte
carve-outs — no `\c`/`\id` strip, no positional NBSP mapping — were applied to any
selection that merely TOUCHED an attribute run. Those carve-outs are earned by one
fact only: the bytes land in a node that stays tagged "attribute", which
`$textNodeTier2Transform` skips. A touching selection loses that the moment its
removal takes the range out of the run, so pasting `\c 5` over one produced a real
chapter node mid-paragraph with the rest of the paragraph stranded outside it — the
save-loop poisoning the strip exists to prevent — and a pasted interior NBSP was
read as display whitespace and saved as a plain space instead of surviving as data.
Measured with the range starting in body text and reaching into the run, and with
it starting inside the run and reaching past the closer.

The carve-outs now apply only to a selection wholly inside one attribute node. The
insertion MECHANISM is unchanged for every touching selection — one `insertText`,
never the paragraph-splitting line replay and never Lexical's rich-paste node
insertion — so a multi-line payload landing in body content this way still collapses
per newline rather than splitting a paragraph the removal has cut a char span in
half in.

The single-paragraph fixture the new pins use is what made this measurable at all:
the earlier probing needed a cross-paragraph range, which does not hold in jsdom.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
…e losses

Deferred item 7 excluded `2sa.usj.ts` from the copy→paste corpus sweep because it
did not round-trip clean when last measured. Re-measured against the same harness:
three classes of divergence remained, two of them clipboard bugs of the family this
round has been closing.

A collapsed note's `category` was dropped. `createNote` builds the `\cat` display
run only for the editable EXPANDED layout — a deliberate view decision — so a
collapsed note has no category bytes on screen for the copy walker to pick up. Same
shape as the note caller, which renders as an empty glyph and is recovered by the
walker for exactly this reason, so the category is recovered the same way, placed
where the file has it (`\f + \cat People\cat*\fr …`, no trailing space: a space
there re-tokenizes into a stray text child inside the note).

A spanning table cell's `colspan` was dropped. USFM tables have no pipe attributes
at all — a cell's width lives in its marker NAME (`\thc3-4`), which the tokenizer
trims off into `colspan` on the way in — and the display rendered the bare marker,
so the editor showed, copied, and re-tokenized a one-column cell.
`tableCellMarkerWithSpan` already existed for the `UnknownNode` byte builder, whose
table branch went dead when cells became `ImmutableTable*` nodes; it is now exported
and shared by both. The committed 2SA Lexical fixtures are regenerated: six lines,
the three spanning cells in editable and visible modes.

The third class is inherent and stays: a verse's `sid` is derived from book +
chapter + verse when ParatextData produces USX, and USFM has no bytes for it
anywhere. Measured rather than assumed — a plain load→save of this fixture keeps all
26 of them, so the loss belongs to the carrier, not the adaptors.

With those two fixed, 2SA round-trips to full USJ equality modulo the 26 verse sids,
so it joins the sweep as a running case rather than a skip: skipping would trade 143
items of real-world coverage for one attribute. The carve-out is asserted on both
sides (the source really carries them, the paste carries none) so it can neither be
vacuous nor outlive the gap, and the re-stamped document USJ version is asserted
rather than waived. The whole sweep file, 2SA included, runs in about a second.

Two gaps the old exclusion reason listed are already closed and were confirmed so:
the empty `\b` blank-line paragraph and the `\ref` target both round-trip.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
…periph title loss

The carve-out is earned by paste-equals-typing, not by the bytes never
re-tokenizing: the "attribute" tag survives insertion, but the caret-departure
settle re-tokenizes the paragraph, so a pasted \c in a value does become a
chapter marker — identically to typing it there. Both halves are now pinned
settled rather than read before the settle, so the residual is visible instead
of looking safe.

Periph's parsed attributes were spread after its title, and the title attribute
is not reserved, so a periph line carrying that attribute silently lost its
title.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
Comment-hygiene sweep of the clipboard/tokenizer work: rewrite backward-facing
comments (dated live-repro narration, "used to"/"previously" change
narration, personal attribution, "this branch" framing) added by this branch
so they read as durable, forward-facing documentation instead of PR
provenance. No logic, assertions, or names changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCjocx9uZcPqB2esPh9r6f
`UnknownNode.isSelected` answered purely on child membership, which a
`NodeSelection` never satisfies — it marks a node by its own key. A
construct selected outright therefore reported unselected, and
`$appendNodesToJSON` hoisted its (also unselected) children, copying
nothing for it. Fall back to the base key-membership answer for a node
selection while keeping the child-membership rule that makes the two
clipboard carriers agree at a range boundary.

`ContextMenuPlugin` is exported on its own but relied on
`ClipboardPlugin` to register the empty-copy guard, so a host mounting
it alone got the placeholder-overwrites-clipboard bug the guard exists
to prevent. Register the guard from both plugins; a duplicate
registration at the same priority is inert.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ention gaps

The `\c`/`\id` strip ran to the next backslash or newline, so a paste of
`x \c 5 y` landed as `x ` — content past the chapter number was destroyed
with no feedback. The tokenizer reads only the next whitespace-delimited
word as the number (getNextWord, usfmFragmentToUsj.ts), so removing more
than that was never required to prevent the second-ChapterNode poisoning.
Split into CHAPTER_TOKEN (marker + separator + one word, stopping at a
backslash) and BOOK_ID_TOKEN, which keeps the wider rule because an `\id`
line's whole remainder genuinely is payload. The pin that had encoded the
loss now asserts the content survives.

`Editor.tsx`'s structureProtectionMode pass-through had no coverage:
Editor and MarkerEditPlugin both default the prop to "off", so deleting
the line left all 4,234 tests green while silently starving
StructureKeyboardPlugin's sanitizer on a protected document's paste.
Threaded the option through mountStandardViewEditor and pinned it against
a real Editor mount.

Also from the review:

- Reuse shared-react's $hasCopyableSelection in $handleCopyForStandardView
  instead of respelling its negation. The two layers must agree on what
  "nothing to copy" means, including the node-selection carve-out, and one
  shared predicate is what keeps them agreeing.
- Derive the four new marker regexes from ENGINE_MARKER_NAME_BYTES. Two
  spellings had already diverged — `[a-z0-9-]i` excludes `_`, so
  `\my_marker` was a marker to the strip and not to the NBSP rules, and its
  separator NBSP survived into content as a literal `~`.
- Collapse isParaKindMarker/isCharKindMarker onto one body. The module
  exists so the two tiers cannot disagree; two copies of the same five
  lines is how they would come to.
- Drop the `$` sigil from normalizePastedNbsp and stripPastedChapterAndBookId.
  Both are pure string functions with no editor-context requirement to
  document, which CLAUDE.md requires of every exported `$`-prefixed one.
- Strip spec-section IDs and provenance narration from comments, per
  CLAUDE.md's rule that code must stand on its own.
- Move the clipboard semantics doc out of docs/superpowers, which
  .gitignore:67 ignores — it was tracked only by a force-add, so edits to
  the durable contract would never have shown in git status. Fix its dead
  cross-reference and the stale "not yet run" row for the e2e spec.
- Pin the Lexical version $selectionToUsfmText mirrors its walk from.

New pins were each verified by reverting the fix and confirming they go
red; the suite is green at shared 697, shared-react 1674, platform-editor
1864, with typecheck, lint and format clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This repository commits each published package's rolled-up build output,
which the eten-tech-foundation repository does not, so the dist inherited
by rebasing onto this main predates every commit on this branch.
Rebuilt with `npm run rebuild-committed-output`.

Also record the `EditorRef.copy()`/`cut()` behavior change in the
changelog. The signatures are unchanged, so a consumer upgrading gets no
compile-time signal that an empty-selection copy no longer writes to the
clipboard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…undaries

In Standard view a read-only construct's display bytes — a figure's
`\fig ` opener, its `|src="…"` attribute run, its `\fig*` closer — are
ImmutableTypedTextNode decorators whose glyph text is an ordinary DOM text
child of the decorator's own element. Lexical 0.43's
$internalResolveSelectionPoint resolves a DOM point inside a decorator's
element to null (the walk up from the text reaches the decorator, which is
not a TextNode), $internalResolveSelectionPoints then nulls the WHOLE
selection, and on commit updateDOMSelection calls removeAllRanges()
because the previous selection was inside the editor. Measured effects: a
mouse drag begun on the `\fig ` glyph left the editor with no selection
at all, so Ctrl+C reached only the empty-copy guard and the clipboard was
never written; a drag from the caption into the attribute run stopped
dead at the caption's end.

DecoratorBoundarySelectionPlugin (shared-react, mounted unconditionally)
handles SELECTION_CHANGE_COMMAND at CRITICAL — Lexical dispatches it even
for a selection it resolved to null, while the DOM selection is still
intact — and moves each offending end to the decorator's boundary in its
parent, the one form Lexical does resolve. A range grows outward so the
glyph is wholly in or wholly out; a caret takes the nearer side. The
snapped DOM points are turned into a Lexical selection in the same update
via $createRangeSelectionFromDom, because moving the DOM selection alone
would still leave a null pending selection for the reconciler to clear.

It only acts when $getSelection() is null (a resolved selection cannot
have an end inside a decorator, and reading the DOM over one would
clobber a programmatically placed caret), and it steps around the note
caller: TrailingNoteCaretGuardPlugin recognizes a landing on a collapsed
note's caller precisely by the selection being unresolved, and supplying
one there disabled that repair and left the caret in the note's hidden
body (measured, pinned).

Pins: six platform tests against a real `\fig` document (five were red
before the plugin), two shared-react guard-rail tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ments, keep paste rules under protection

Three hand-QA findings against Standard-view copy/paste, all rooted in
which clipboard carrier a consumer reads.

Paratext 9 paste is HTML-FIRST (IHtmlEditorCopyPaste.cs: the CF_HTML
fragment is pasteHTML'd whenever present, and Standard-Reverse.xslt reads
the fragment's text() as USFM). Standard view's text/html was Lexical's
DOM export, which is lossy in two independent ways:
ImmutableNoteCallerNode.exportDOM carries a collapsed note's caller only
as a data-caller attribute, and UnknownNode.exportDOM returns
{element: null}, so figures, sidebars, periphs, refs and optbreaks are
absent from the html altogether. A P10→P9 paste of
`\x - \xo 1:26: \xo*\xt 1Cor 11:7\xt*\x*` therefore arrived as `\x \xo …`
with no caller and rendered nothing. text/html is now the same USFM bytes
as text/plain (usfmToClipboardHtml: HTML-escaped, one
`<p><span style="white-space: pre-wrap;">…</span></p>` per line — the
block shape Lexical's own TextNode export produces, so this editor's own
html re-import path still sees one block per line). invertDisplayNbspInHtml
is gone with the DOM export it normalized. The internal
application/x-lexical-editor flavor is unchanged.

P9's own text/plain reduces a note to the caller glyph it displays (`a`,
or `*` for a `-` caller); the note's bytes ride the html as an escaped
`<!--usfm:…-->` comment inside a span P9 marks `exclude`. getPastePayload
preferred text/plain, so a P9→P10 footnote paste inserted a literal `a`.
paratext9HtmlToUsfm ports Standard-Reverse.xslt over the DOM (text,
usfm: comments with %XXXX unescaping, exclude/include modes, div/tr/br
newlines, span.attribute, usfmopen/usfmclosed with the nested `+`) and
wins over text/plain whenever the html carries P9's signature — a usfm:
comment, or a usfm_<name> class together with usfmopen/usfmclosed, which
no word-processor, browser or P10 html carries.

Under structure protection (Simple interface mode) $handlePasteForStandardView
declined every paste to StructureKeyboardPlugin's html sanitizer. That
carrier could not express a figure (the loss the hand QA saw: the same
clipboard pastes a `\fig` in Power mode and nothing in Simple mode), it
skipped stripPastedChapterAndBookId (a protected paste of `\c 7` created a
second chapter node, the save-loop poisoning the strip exists to prevent),
it skipped the positional NBSP rule, and it would have ignored the P9
decode entirely. The marker engine has no protection gate — a `\v` or `\p`
byte tokenizes into a real marker in either mode, typed or pasted — so the
sanitizer never protected Standard view against marker bytes, only against
verse/para NODES in a DOM import that Standard view's html no longer
produces. The claim now handles a protected document with the same bytes
as an unprotected one, except that it declines a selection
$shouldBlockSelectionReplacement refuses (so that refusal keeps one owner)
and never replays paragraphs: a multi-line payload joins with single
spaces, the sanitizer's own convention for a removed paragraph boundary.
Exposes structureKeyboard.utils from shared-react's index for that
predicate; shared-react is private, so nothing new reaches the published
surface.

Pins: paratext9Clipboard.utils.test.ts (18), the html/plain parity
describe in clipboardCopyFidelity, the P9→P10 paste describe in
markerPasteFidelity, protectedFigurePaste.test.tsx (12 — the same-editor
copy builds the figure under protection, `\c`/`\id` are stripped in both
modes, the P9 comment decodes under protection, a legacy DOM-export html
inserts prose and no figure), and the structure-protection describe in
whitespaceDisplay.plugin.utils.test.tsx, whose Editor→plugin wiring pin
stays falsifiable as a mode difference (a two-line paste yields one
paragraph protected, two unprotected). Semantics doc and changelog updated
to state the new contracts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…to a decorator

Rewriting the DOM selection to the decorator's boundary was the part a real
browser rejected. Chromium extends a mouse drag only from a base it placed
itself: when a press lands in a `\fig ` glyph, the browser's own caret sits
in the glyph's text, and replacing it with the scripted element point beside
the decorator meant every subsequent mouse move re-placed a collapsed caret
under the pointer — no range ever formed, and with a caret Ctrl+C fires no
copy event at all. (Chromium also clamps a drag begun inside a
`contenteditable="false"` island to that island, so a drag from the glyph
selects at most the whole figure; that is the browser's rule, not ours.)

The snap now builds the Lexical selection from the snapped points directly
(a selection-shaped object handed to $createRangeSelectionFromDom, which
reads only the four anchor/focus fields) and, while a pointer button is
down, marks it clean so the commit leaves the browser's DOM selection
exactly where the browser put it. Raw interior DOM points are confined to
the drag: on pointerup/pointercancel the snapped selection is marked dirty
in an update whose event type Lexical does not re-derive from the DOM, so
the boundary form is written out through the ordinary reconciler path; a
snap with no button down (Shift+Arrow, programmatic) writes immediately.
Without that materialization the next event-less editor.update() would
re-read the interior points, resolve null, and its commit's
removeAllRanges() would take the visible selection with it.

SKIP_DOM_SELECTION_TAG was not usable for the drag case: a selection-only
repair dirties no nodes, so the tag survives into the next commit and
suppresses the DOM write for an unrelated caret move (measured).

Pins: the DOM points stay untouched while the button is down, are
materialized on release, are written immediately with no button down, and
the selection survives an event-less update afterwards; the copy pins now
dispatch a real `copy` event so the update clones the state selection the
way a browser copy does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tjcouch-sil and others added 6 commits September 22, 2026 12:00
Paste:
- A payload the `\c`/`\id` strip empties is claimed and changes nothing.
  It used to delete the selection and insert nothing, both in the
  Standard-view claim (and its attribute-context branch) and in the in-note
  `\fp` claim, which now also decides "multi-line" on the stripped text.
- A line the token fills but for its own trailing separator is dropped: a
  chapter line copies as `\c 5 `, and the leftover space became an extra
  paragraph when pasted back.
- Every member of a leading NBSP run maps to a space (`  word`
  pasted as ` ~word`).
- A Paratext 9 fragment that decodes to nothing no longer wins over its
  own `text/plain`.
- No html is parsed unless the carrier choice reaches it: the P9 decoder
  checks its signature bytes before parsing, and the generic html decode is
  lazy. A word-processor paste went through six DOM parses.

Copy/cut:
- The internal flavor is omitted for a selection touching a book or
  chapter line. A native paste (Shift+Insert) rebuilt the copied
  ChapterNode from it past the strip, and every later save failed.
- A selection starting at a collapsed note's caller carries no leading
  space; the `\id` line's separator copies as a space, not an NBSP.
- The empty-copy check keys on `text/plain`: the internal flavor of a
  byte-less range is `{"nodes":[]}`, a non-empty string.
- A structure-protected cut of a selection StructureKeyboardPlugin refuses
  is declined to it (as paste already was); it used to remove the range.

UnknownNode.isSelected answers false for a collapsed caret in a child.

Docs: clipboard-semantics.md records each rule, the text/html blank-line
limit, the P9 note-NBSP limit, and hidden-marker views' html. The doc and
a test comment now state the condition under which P9's NormalizeTokenUsfm
breaks the line before a Paragraph token, rather than "every" one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The CRITICAL cut handler arms the whole-paragraph reap from the pre-cut
selection before any claim runs. When the Standard-view cut claim declines
a protected selection to StructureKeyboardPlugin, nothing changes, so no
commit runs the update listener that resets the arm, and the next commit's
transforms would read it as a fresh whole-paragraph deletion.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Positional NBSP normalization deleted every NBSP right before a marker
token, to remove the spacer a note's display layout puts before each
child marker and before `\f*`. That also deleted spaces users wrote:
`the<NBSP>\nd Lord` from a word processor pasted as "theLord", and
`\nd Lord<NBSP>\nd*` lost the span's trailing space.

Such an NBSP now becomes a plain space, like one after a marker. The text
cannot tell a layout spacer from a space typed in the same place, and a
stray space can be deleted where a lost one cannot be recovered. The only
copy that still puts that layout on the clipboard is the read-only Markers
view's stock copy, which is not valid USFM on other counts either (no
caller, no separator after a char marker); Standard view's own copy never
carries a spacer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… items

In Standard view a milestone's glyphs and attribute text ride in an
AttributeRunNode sibling, as do a verse's \va/\vp and a chapter's \ca.
$shouldIgnoreNodeForContentIndexes skipped every other display-only node
but not that wrapper, so every USJ position after one of them in the same
parent was one content item too far — for annotations and for
$getUsjSelectionFromEditor / $getRangeFromUsjSelection alike. The
editor-to-USJ conversion already skips the wrapper wholesale; the logical
content model now agrees with it (pinned in the content-agreement suite).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Markers view (markerMode "visible") used Lexical's own copy, which
writes the display text. That is not USFM there: no note caller, no space
after a char marker, no verse or chapter numbers, no figure markers, and
the note layout's NBSP spacers. Pasting it produced broken USFM, and it
was the last copy path putting those spacers on a clipboard.

MarkersViewCopyPlugin maps the selection to USJ locations, rebuilds the
document as Standard view in a throwaway editor, and runs the Standard-view
copy walker over the same range, so both views write the same bytes. It
writes text/plain and text/html only: the internal flavor would carry
display-shaped nodes. A cut in a read-only editor copies and removes
nothing. The clipboard write is shared with Standard view's handler
($writeCopyPayload).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@lyonsil lyonsil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep review of the clipboard/paste changes

This reviewed the copy/paste USFM-fidelity changes against the current head, focusing on the newest commits since the last human review round. 22 findings, each checked against the code and, where a fix was proposed, checked that the fix actually works. Each comment's header states the severity the reviewer settled on and carries the status checked and confirmed — verified against the code as described.

A few findings below anchor pre-existing code that this PR doesn't touch but that its own new code newly exposes; those land here in the body rather than as inline comments, since GitHub only allows inline comments on lines this PR's diff actually changed.

Findings that could not be attached to a line in this diff

#2 (high · checked and confirmed) libs/shared-react/src/plugins/usj/annotation/selection.utils.ts:436 - Selecting a range that starts right before a character-styled span's (or footnote's) own opening marker glyph and ends right before its own closing marker glyph, in the read-only Markers view, copies nothing to the clipboard with no error shown — an ordinary selection spanning two different \v verse markers in the same paragraph does not trigger this, since verse numbers are decorator nodes this code path never treats as marker glyphs.

What happens: $getLocationFromNode (libs/shared-react/src/plugins/usj/annotation/selection.utils.ts:436-441, pre-existing, not touched by this PR) resolves an element-type selection point landing on a VisibleMarkerNode child to a location keyed only on the parent node, ignoring the child's offset — so two different offsets in the same paragraph (e.g. selection ends framing two different \v markers) produce identical start/end USJ locations. $selectionToUsfmViaStandardView's !range?.end check (packages/platform/src/editor/MarkersViewCopyPlugin.tsx:53, new this PR) doesn't catch this since range.end is a valid, non-undefined object — it's just equal to range.start. The rebuilt selection in the throwaway Standard-view editor is therefore collapsed, $selectionToUsfmText returns "", and $writeCopyPayload's empty-payload check silently swallows it.

Why it matters: Not an edge case — any ordinary multi-verse paragraph has this shape, and a drag or triple-click selecting a whole verse can easily land both ends on marker-glyph boundaries. The user gets a visibly successful copy with a clipboard that still holds whatever it held before, no error, no fallback. This is a pre-existing imprecision in shared selection-mapping code that this PR's new Markers-view copy feature is the first caller to expose through a user-visible silent failure.

Fix: In the $isVisibleMarkerNode(childAtOffset) branch (selection.utils.ts:436-442), check whether childAtOffset is the opening or closing glyph the same way $findMarkerNode does (text.endsWith("*")), and return a UsjClosingMarkerLocation (with a closingMarkerOffset) for the closing glyph instead of always returning an offset-less UsjMarkerLocation, mirroring the $isMarkerNode branch immediately above it. Add the regression test against a CharNode (or note) built with its own opening and closing marker children in markerMode: "visible" — not against \v markers, which this code path never matches.

How this was checked: Verified with a runnable probe: a selection from right before a marker glyph's opening ImmutableTypedTextNode to right before its closing one produces identical start/end USJ locations from $getLocationFromNode (selection.utils.ts:436-441), because it keys only on the parent node. This reaches MarkersViewCopyPlugin.tsx:53's !range?.end check (doesn't catch it), collapses the rebuilt selection, and $writeCopyPayload silently no-ops. $getLocationFromNode is pre-existing (predates the merge base) and is also reached by pre-existing OnSelectionChangePlugin/EditorRef.getSelection() call sites, so this PR sharpens an existing latent defect into a visibly-empty clipboard rather than being the first exposure of it.

Not attached to a line: no longer on a changed line; cannot be posted inline

#4 (high · checked and confirmed) packages/platform/src/editor/markerEdit/MarkerEditPlugin.tsx:876 - A Cut that structure protection refuses in a non-Standard editable view (e.g. Unformatted) can leave a stale paragraph-deletion arm that later reaps an unrelated, untouched paragraph.

What happens: The clear-the-arm fix for a refused protected Cut (context.wholeParaDeleteExpected?.clear() at line 913) lives only inside the isStandardView-gated plugin block (lines 876-944). But the CRITICAL handler that unconditionally arms wholeParaDeleteExpected (lines 945-956, pre-existing, not gated on isStandardView) and StructureKeyboardPlugin's own CUT refusal are both fully active in any markerMode: "editable" view, Standard or not. In a view with isStandardView false but markerMode: "editable", a Ctrl+X spanning a paragraph boundary arms the reap set, gets refused by StructureKeyboardPlugin, and nothing registered for that view clears it.

Why it matters: Same phantom-paragraph-reap outcome as item #5, reachable in a second view mode. Platform.Bible's own current production menu does not expose an Unformatted view (per a prior review comment on this PR, MarkerEditPlugin.tsx:437), but this package's own public view prop documents "unformatted" as a supported mode (editor.model.ts:632), and the demo app's dev harness (demos/platform/src/app/app.tsx, via ViewModeDropDown) wires it directly into the real <Editor> with structure protection selectable independently. So this is reachable through the package's own documented API and dev tooling today, even though Platform.Bible's shipped menu doesn't currently surface it — the prior comment settles a different, narrower question (what Platform.Bible's menu exposes) than this finding depends on (what the package itself allows).

Fix: Scope the arm's lifetime to the originating gesture's dispatch, not to Lexical's commit count: stamp each armed key with an id bumped once per top-level KEY_DOWN_COMMAND/CUT_COMMAND dispatch (before any tier runs), and additionally schedule the arm's own removal via queueMicrotask/Promise.resolve().then() at arm time, so an arm no consuming commit claims within the synchronous handling of that one keystroke/cut event self-destructs before any later, unrelated commit's transform pass can observe it. This works for every view and every gesture without a per-call-site clear.

How this was checked: The clear-the-arm fix (MarkerEditPlugin.tsx:913) only runs inside the isStandardView-gated CUT_COMMAND block. The CRITICAL-priority handler that unconditionally arms wholeParaDeleteExpected (lines 945-956) is pre-existing and not gated on isStandardView, so it also arms in a non-Standard editable view. A refused cut produces zero real commits, so the update listener that clears the arm never runs. Reachability: even though a prior review comment says Platform.Bible's own production menu doesn't currently expose the Unformatted view, this package's own public view prop (editor.model.ts:632) documents "unformatted" as a supported mode, and the demo app's dev harness wires it directly into the real with structure protection selectable independently — so this is reachable through the package's own documented API, not merely hypothetical.

Not attached to a line: no longer on a changed line; cannot be posted inline

#5 (high · checked and confirmed) packages/platform/src/editor/markerEdit/MarkerEditPlugin.tsx:987 - A Backspace or Delete keystroke that structure protection refuses can cause an unrelated, later paragraph to silently vanish or reset.

What happens: MarkerEditPlugin's KEY_DOWN_COMMAND handler (unconditional, not gated on view — lines 987-1005, pre-existing) calls $armWholeParaDeletion/$armCollapsedParaDeletion for Backspace/Delete, then returns false. StructureKeyboardPlugin's own KEY_DOWN_COMMAND handler (registered after, same COMMAND_PRIORITY_HIGH) then blocks the keystroke via $shouldBlockStructuralEdit and mutates nothing. Because nothing was dirtied, Lexical never commits, so the registerUpdateListener that clears both arm sets (lines 1146-1147) never fires. The armed paragraph key(s) survive into a later, unrelated commit; if that paragraph is ever transiently emptied by a routine rebuild (an explicitly normal occurrence per markerEditDeletion.utils.ts's own comments), $paraMarkerDeletionTransform reads the stale arm as fresh provenance and reaps/resets a paragraph the user never touched.

Why it matters: This is the same silent-data-loss class this PR's own de737c0a commit fixed for Cut (see item #7's discussion of MarkerEditPlugin.tsx:913) — but only for Cut. Backspace/Delete is the far more common gesture and has zero test coverage (grep for wholeParaDeleteExpected/collapsedDeleteCaretParas across every test file: zero hits).

Fix: Same corrected mechanism as item #4's rewrite: tag the arm with a per-dispatch id (or self-expire it via a microtask scheduled at arm time) so it cannot outlive the synchronous handling of the keystroke that created it, rather than trying to key it to Lexical's commit count.

How this was checked: MarkerEditPlugin.tsx:987-1005 (pre-existing, unconditional, HIGH priority) arms wholeParaDeleteExpected/collapsedDeleteCaretParas on Backspace/Delete in every editable view, mounted before StructureKeyboardPlugin. When StructureKeyboardPlugin's own KEY_DOWN_COMMAND handler refuses under "protected" mode without touching either arm Set, no commit happens, so the update listener that clears the arms never runs. This reproduces even in Standard view itself — it needs no view-mode caveat — and is the more commonly hit half of the same bug class this PR's own commit fixed only for Cut.

Not attached to a line: no longer on a changed line; cannot be posted inline

(AI-assisted, with my guidance)

Comment thread packages/platform/src/editor/Editor.tsx Resolved
tjcouch-sil and others added 6 commits September 22, 2026 14:24
…ange

Three handlers ran ahead of StructureKeyboardPlugin's own refusal and committed
to something it was about to refuse.

The in-note multi-line paste claim registers at CRITICAL and removes the
selected range before it decides how to replay the lines, so a protected
document pasted over a selection reaching out of a footnote across a verse
marker lost the whole range — the new test shows the document reduced to the
pasted text. It now consults the same predicate first.

The paragraph-reap arms were set from the pre-gesture selection with no such
check. A refused keystroke or cut mutates nothing, so nothing commits and the
update listener that resets the arms never runs; the arm survived into the next
unrelated commit, where a paragraph transiently emptied by a rebuild read as
this gesture's provenance and was reaped. Guarding both arm sites covers every
view and both gestures, so the Standard-view-only clear on the cut path goes.

StructureKeyboardPlugin's CUT refusal moves to CRITICAL: a refusal has to
outrank the actor it refuses, not tie with it. At HIGH the winner was mount
order, an incidental property of JSX position. KEY_DOWN stays at HIGH — it has
no same-priority actor and MarkerEditPlugin's own KEY_DOWN handler must keep
running ahead of it.

Also gates the in-note claim on a raw newline before the strip/normalize pass,
which neither can introduce, so the common single-line paste stops paying for a
result it discards.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An element selection point landing on a read-only marker glyph resolved to a
location keyed only on the parent node, so a point before a char span's opening
glyph and a point before its closing glyph produced the same USJ location. A
range framing the whole span therefore rebuilt collapsed, and the Markers view's
USFM copy wrote nothing while still claiming the event — a copy that looked like
it worked and left the clipboard holding whatever it held before.

The closing glyph now emits a UsjClosingMarkerLocation, told apart by the same
trailing-`*` test $findMarkerNode resolves the two back with.

Reachable from ordinary gestures: a click in the leading half of one glyph
followed by a shift-click on the other. A single drag is not affected —
DecoratorBoundarySelectionPlugin deliberately grows the later end outward.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… \c survives

$writeCopyPayload now owns the "never remove text from a non-editable editor"
rule instead of leaving it to one of its two callers. ClipboardPlugin's Ctrl+X
listener is bound to the root element in every view and dispatches CUT_COMMAND
with no editability check, so a read-only Standard view reached the removal with
nothing but the call site's discipline in the way.

stripPastedChapterAndBookId only drops a whole line when the token OPENS it.
Whitespace the user wrote ahead of the token is content, and the widened
blank-remainder check took it — and the line break — along with the token.

Also pins the `\va*` + NBSP + `\vp` case: the NBSP becomes a space, which the
tokenizer reads as content and which therefore unfolds `\vp` into a standalone
span. That is what Paratext 9 produces from the same bytes with AllowInvisibleChars
off, and dropping the byte to keep the fold would lose what the user pasted.

Drops the lazy-html thunk `||` already provides, uses the codebase's ancestor
walk in $selectionTouchesBookOrChapter, and adds the `typeof event === "object"`
guard the analogous duck-checks keep before `in`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…n cleanups

formatPara and insertNote were the only content-mutating EditorRef methods on a
deferred commit, so a caller that copied straight after one read the document as
it was before the call. They join the rest on `discrete: true`.

A visible marker mode left editable now says so: the marker-edit engine that
heals a damaged glyph is gated to "editable", so that surface takes ordinary
edits with no repair behind them.

MarkersViewCopyPlugin moves into alphabetical position among the plugin
children; the Paratext 9 signature regex is built from the constants its precise
check already uses; and the new decorator-boundary tests build their trees the
way the style guide asks.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`normalizePastedNbsp` ran the after-a-marker and before-a-marker rules as two
passes with two token regexes built from the same MARKER_TOKEN, substituting the
same byte. They are now one alternation over MARKER_ADJACENT_NBSP, so the two
directions cannot drift apart in what they recognize or in what they emit, and
the capture-plus-backreference that only re-emitted the token it wasn't changing
is gone.

Equivalence checked exhaustively before the change: over every string up to
length 6 from an alphabet covering each structural element (backslash, `+`, `*`,
name bytes, NBSP, space, newline, pipe, plain text) plus the realistic USFM
fixtures, 5,229,057 inputs, the two forms agree on every one.

Lookbehind matches the existing use in this file and the repo's es2023 target.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@tjcouch-sil

Copy link
Copy Markdown
Member Author

Thorough review — thanks. 17 of the 22 fixed, 5 declined, each with a reply on its thread. Pushed as d4eb5d3c, 187f24df, f56ece0c, 2ac599b1, 3b5f048c (dist), 9dd93da5.

Gates: typecheck, eslint and prettier clean; all three suites green (shared 701, shared-react 1671, platform 1955); verify-committed-dist.mjs passes. Every behavioural fix below has a test that was made to fail against the unfixed code first.

The three that could not be attached to a line

#2 — fixed (187f24df), and reachable. Verified end to end, including the gesture question your writeup left open: a single drag does not reach it (DecoratorBoundarySelectionPlugin deliberately grows the DOM-order-later end outward), but a plain click in the leading half of one glyph followed by a shift-click on the other lands both ends as element points on the CharNode, which is an ordinary "adjust my selection" gesture. $getLocationFromNode now emits a UsjClosingMarkerLocation (closingMarkerOffset: 0) for the closing glyph, told apart by the same trailing-* test $findMarkerNode resolves them back with — your fix direction, with the offset value it did not specify.

Two pins: the unit case in selection.utils.test.ts, and a Markers-view copy of a span framed by its own glyphs, which returns '' without the fix — the silent-empty-clipboard symptom exactly.

#4 and #5 — fixed (d4eb5d3c), by a different mechanism than proposed. The diagnosis holds in both cases; the remedy does not. The arm is deliberately consumed in a later dispatch than the one that sets it — KEY_DOWN arms, then the browser's own beforeinput drives the deletion in a separate event — so neither a per-dispatch id nor a queueMicrotask self-expiry at arm time can work: both would clear the arm before the deletion it exists to attribute, breaking whole-paragraph deletion outright.

What works is simply not arming for a gesture structure protection will refuse, asked with the same predicates StructureKeyboardPlugin decides with ($shouldBlockStructuralEdit for a keystroke, $shouldBlockSelectionReplacement for a cut). That covers Backspace, Delete and Cut, in every view, and makes the Standard-view-only clear this PR had added redundant — so it is gone, which is also what #7 asked for.

Pinned in paraWholeDeletion.test.tsx: a refused cross-paragraph Delete under protected, then the programmatic emptying the transient-emptiness contract already covers. Without the guard the untouched paragraph is reaped; with it, it survives. Worth noting the test had to dispatch the keystroke as its own update — placing the selection in the same update commits it, and that commit's update listener resets the arms, hiding the window entirely.

The five not taken

#8 (P9 parity — see thread), #13 (bounded by one chapter; its own piece of work), #14 (no demonstrated path, your own bar for closing it), #20 (unreachable today; wants its own tests), and — the one that was taken after re-examination — #19 is merged in 9dd93da5, since its only stated blocker was #8.

Still outstanding

The clipboard round-trip E2E in paranext-core#2656 has not been re-run against this build yet; doing that next, since #7 changes cut priority and #6 changes note-insertion commit timing, both of which that spec exercises.

(AI-assisted)

@tjcouch-sil

Copy link
Copy Markdown
Member Author

The clipboard round-trip e2e (paranext-core#2656) has now been re-run against this branch at 9dd93da5 — green in 3.3 minutes on the first attempt, no retry, every step including the reworked drag and undo steps. That closes the one item I left outstanding above: #7's cut-priority change and #6's discrete note insertion are both paths that spec drives.

(AI-assisted)

@lyonsil lyonsil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up on #4 and #5 (stale paragraph-reap arm)

This re-checks the #4/#5 fix in d4eb5d3c against head 9dd93da5. The rest of the previous review's findings are settled and their threads are resolved. There are two findings, one on each arm site TJ changed, and both are now on changed lines so they sit inline. The severity shown is the one the reviewer settled on, and checked and confirmed means the finding was reproduced against the code at this head.

In short: the guard is correct for structure protection's refusals, but those are not the only refusals that reach the arm. Two others still leave a stale arm behind, and the mechanism the earlier reply ruled out works in this editor.

(AI-assisted, with my guidance)

tjcouch-sil and others added 3 commits September 22, 2026 17:42
getFirstChild()/getLastChild() return `T | null`, and requireDefined only
narrows away `undefined`, so the spec typecheck failed on all three CI
runners (TS18047). Coalesce the null to undefined first, the same way the
surrounding tests already do.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The paragraph-reap arms consulted only structure protection's refusal. Two
other refusals claim a Backspace/Delete after the engine has armed and
commit nothing, so the arm survived into the next unrelated commit and a
transiently emptied paragraph was reaped: the opaque-block guard (a
selection ending inside a figure caption, in every protection mode) and
guarded mode's first two-step press over a verse marker. The delete-key
arms now also expire on a microtask. That is safe because Lexical's
rich-text delete keys dispatch and apply the delete synchronously inside
the same KEY_DOWN dispatch, and the paragraph transform consumes the arm in
that update.

The cut arm cannot expire that way: Lexical's own cut, which Unformatted
view uses, removes the range only after `await copyToClipboard(...)`, so a
microtask expiry would drop the arm before the removal and leave a whole
cut paragraph as an empty husk. It asks the opaque-block guard's predicate
instead, the one refusal at its priority it did not already consult. A new
test pins the Unformatted-view cut so the expiry never spreads to it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@lyonsil lyonsil left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@lyonsil reviewed 63 files and all commit messages, made 1 comment, and resolved 40 discussions.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on tjcouch-sil).

@tjcouch-sil
tjcouch-sil merged commit c5c4f37 into main Sep 23, 2026
6 checks passed
@tjcouch-sil
tjcouch-sil deleted the standard-view-pt-4201 branch September 23, 2026 14:29
tjcouch-sil added a commit that referenced this pull request Sep 24, 2026
…sorbs-text

Resolution: main's Standard-view external-paste handler now claims external
pastes at HIGH and replays them as typed, so the chapter line's LOW
"insert as one line" claim no longer ran for them. The chapter line now uses
main's own pipeline instead of a second one:

- The insertion tail of `$handlePasteForStandardView` is extracted, unchanged,
  as `$insertPastedText` (strip `\c`/`\id`, normalize NBSPs, replay lines as
  typed, or join them in a protected document).
- A paste or drop on a chapter line goes through `$insertPastedText`, so its
  first line goes onto the chapter line and each later line starts a paragraph
  after it, exactly as typing it there does.
- The chapter-line paste claim moves to CRITICAL, ahead of Lexical's rich
  paste and structure protection's html sanitizer. Both insert nodes at the
  caret, and a chapter line has no block for them: an internal paste in a
  protected document threw and did nothing.
- `getDataTransferPayload(dataTransfer, namespace)` reads a drop with the same
  carrier rules as a paste, Paratext 9 html decoding included.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
tjcouch-sil added a commit that referenced this pull request Oct 1, 2026
Brings in #13 (Standard-view copy/paste), #21, and the \periph, gutter
and font-fallback work. Conflict resolutions: node.utils.ts ours;
unknownUsfm.utils.ts both; usfmFragmentToUsj.ts main's pushTableCell
(tests: both sides); selection.utils.ts ours (subsumes main's closing
glyph fix, whose tests pass unchanged); Editor.tsx imports union and
insertNote live-resolution inside main's discrete update;
markerEdit.test-helpers.tsx both; settledGetUsj.test-helpers.tsx both
mount options; dist ours (rebuilt at the end of the round).

Details:
- usfmFragmentToUsj.ts: main's five hunks; this branch's two auto-merged
  comment hunks (the cell-marker regex doc and the para-case table
  comment) are kept. Both call-site conditions this branch pinned
  (`tableEligible && tableRow`, `!token.isNested && !note &&
  !isNoteContext && tableRow`) are present in main's code.
- usfmFragmentToUsj.test.ts: both describe blocks whole (this branch's
  default-stylesheet cell rows, main's \periph rows); no row appears on
  both sides; the doubled defaultStyleInfo import is dropped.
- Editor.tsx imports: copySelection, cutSelection and
  DecoratorBoundarySelectionPlugin added. DeltaOnChangePlugin and DeltaOp
  are not, since this branch no longer uses either.
- settledGetUsj.test-helpers.tsx: structureProtectionMode passes through
  editorWith's options next to view, isReadonly and styleInfo.
- dist: packages/platform/dist taken from this branch whole, index.d.ts
  included.
- Semantic merge fixes (this branch made the view options explicit):
  MarkersViewCopyPlugin.tsx passes its viewOptions to
  $getUsjSelectionFromEditor and the Standard view options to
  $getRangeFromUsjSelection; main's new selection.utils.test.ts row calls
  $getUsjSelectionFromEditor(undefined).
- Lockfile: main adds postcss to platform devDependencies; installed with
  `pnpm install --frozen-lockfile`.
- New annotationLocations/exactAnnotationLocations.test.tsx pins that a
  pasted copy of an annotated attribute run holds no annotation on both
  internal-paste paths that insert nodes: Lexical's rich-paste insertion
  (structure protection off), which reaches the strip through
  SELECTION_INSERT_CLIPBOARD_NODES_COMMAND (the row fails with the strip
  disabled), and the structure-protection html sanitizer, whose nodes come
  from the USFM-bytes text/html and carry no node state. The Standard-view
  external-paste claim, the attribute-context claim, the in-note line
  replay and the char-stack line replay all insert re-tokenized USFM
  text, which carries no node state, so they need no row.

Tests: shared 752 -> 778 (main +26); shared-react 1769+1 skipped ->
1786+1 (main +17); platform 2431 -> 2702+3 skipped (main +269 and 3
skipped, new pin rows +2); utilities 59, scribe 2 unchanged. This
branch's own delta over the merge base: shared +77, shared-react +115,
platform +739. nx lint 10/10, nx typecheck 10/10.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: 2c4fe30c-6a8b-42a8-ba57-9141bb0ab91f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants