Skip to content

PT-4197: Data-fidelity audit, for release-prep (#29) - #51

Merged
lyonsil merged 1 commit into
release-prepfrom
release-prep-29
Nov 4, 2026
Merged

PT-4197: Data-fidelity audit, for release-prep (#29)#51
lyonsil merged 1 commit into
release-prepfrom
release-prep-29

Conversation

@tjcouch-sil

@tjcouch-sil tjcouch-sil commented Nov 3, 2026 •
edited by lyonsil
Loading

Copy link
Copy Markdown
Member

Brings #29 (PT-4197: data-fidelity audit) onto release-prep so it can go into the next RC build. Companion to paranext/paranext-core#2987 (the pick of paranext/paranext-core#2922), which builds against this branch (core's release-prep pins scripture-editors release-prep), so merge this one first.

Testing: this tree (#29 without #42/#45) was never tested on main, so I ran on this branch pnpm nx run-many -t typecheck lint and -t test (--skip-nx-cache) for shared-react, platform-editor and scripture-utilities: all passed (lint: 0 errors, 5 warnings; tests: 8597 passed, 57 expected failures, 0 failed). CI reruns here.

🤖 Generated with Claude Code

This change is Reviewable

@tjcouch-sil tjcouch-sil mentioned this pull request Nov 3, 2026
Merged
@tjcouch-sil @claude @lyonsil
PT-4197: Data-fidelity audit — composed round trip, USX-first net, co…
4bb23a9 
…rpus (#29)

* feat(utilities): let usjToUsxString stamp a caller-specified USX version

usjToUsxString(usj, { usxVersion }) stamps the requested version on the
root element instead of always writing USX_VERSION. USJ cannot carry the
source USX version (the editor regenerates usj.version everywhere), so a
host that needs its USX back at the version ParatextData served it must
say so explicitly. Default output is byte-identical to before.

Part of the PT-4197 data-fidelity audit; lets paranext-core drop its
post-hoc version="3.1" string replacement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: composed PDP round trip, USX-first net, corpus extension (PT-4197)

Pins the full load/save trip over the real per-chapter USX the paranext-core
PDP produces (vendored under testUsfmCorpus/pdp-usx/):

- corpus-pdp-usx-composed-round-trip: USX -> USJ -> mounted editor -> USJ ->
  USX per capture, in Standard, formatted and paragraph-structure views, with
  a pinned expected-divergence list (an unlisted divergence fails; a listed
  one that stops happening fails).
- testusfm-usx-usj-usx-round-trip: the USX-first converter net, so an
  asymmetric usx<->usj pair cannot hide from both directions.
- corpus-testusfm-round-trip and corpus-transform-fixed-point now run every
  fixture in all three editing views (they were Standard-only).
- New fixtures testUSFM-2SA-4 (NBSP shapes, verse bridges/segments, \fe, \x,
  \cl/\cd, callers, \ref), testUSFM-2SA-5 (damaged-but-at-rest: nameless
  markers, double \ca, chapter-line text) and testUSFM-1SA-1 (book intro),
  with .usj derived by the documented recipe (provider USX + usxStringToUsj;
  2SA-5 hand-corrected to the intended marker:"" shape).

Known losses are routed through each suite's expected-failure mechanism,
attributed to scripture-editors #11, #19 or PT-4391, so a merge that fixes
one flips the pin visibly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: drop the pins SE #11 fixed (PT-4197)

SE #11 fixed the leading-NBSP strip in a char span, the double-\ca fold in
the tokenizer, and the nameless-marker converter bug, so their pins flipped
as designed. The 2SA-5 .usj no longer needs a hand edit: the converter now
writes exactly what it holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: keep an authored lone-NBSP content string outside Standard view (PT-4391)

Outside Standard view a data NBSP is held as a literal NBSP, so a content
string that is exactly one NBSP (`\nd ~\nd*`, `\p ~`, a `~` between two
spans or after a note) was byte-identical to the engine's lone-NBSP display
nodes and the editor->USJ conversion dropped it, already at load.

The conversion now drops a lone NBSP only in the slots the engine's display
nodes occupy: right after an editable glyph, right after a read-only glyph
inside a span, first in a span or note, after a collapsed note's caller or
span, or inside an attribute run (tagged/token nodes and the mark cases are
unchanged). The one slot an author's `~` shares with a display node - a
span's first content in the marker modes without a separator, where the
empty-span placeholder sits - is disambiguated by a new node state,
`authoredNbspState`, that the forward adaptor's createChar sets.

The same rule is the live `$isLoneNbspContent` (node.utils.ts), used by the
logical content model and the Tier-2 fragment flattening so annotation
positions and settles agree with the export. CharNodePlugin no longer treats
an authored `~` as an empty span's placeholder when text is typed beside it.

Flips the pinned drops: constructed-usj-round-trip's paragraph-leading lone
NBSP, the 2SA-4 entries in the composed PDP round trip and the testUSFM
round trip (adaptor and transform legs), the formatted-view leading-NBSP pin,
and the logical-model spacer pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: one shared registry of editor views and one mountInView (PT-4197)

Every per-view list the editor suites iterate is now a projection of
EDITOR_VIEWS in editorViews.test-helpers.tsx: the location oracle's
ORACLE_VIEWS/oracleView, the corpus suites' ROUND_TRIP_VIEW_CONFIGS and
EDITING_VIEW_CONFIGS (moved out of corpus-data.ts, which is compiled
with the library and so cannot import a test helper), and the new
EDITING_VIEWS (standard, formatted, paragraph-structure).

The two private Editor mounts (settledGetUsj's mountEditor and the
location oracle's mountInView) merge into one mountInView(usj, view,
opts); mountStandardViewEditor/mountExpandedNoteEditor are thin
wrappers. Also adds footnotePopoverView(host) (mirrors paranext-core's
FootnoteEditor options), forEachView, knownFailureRunner, and
noteEditorRef's optionsInView(view).

The Standard-with-expanded-notes round-trip label is now its registry
name, standard+expandedNotes (was standard-expanded).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: run the position oracles in formatted and paragraph-structure (PT-4197)

logicalModel.oracle gains formatted and paragraph-structure (no
separators); canonicalLocations gains paragraph-structure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: run the footnote popover round trip over every editing host (PT-4197)

note-ops-popover-roundtrip mounts its host in Standard, formatted and
paragraph-structure, and its popover with footnotePopoverView(host), the
options paranext-core's FootnoteEditor builds (so the Standard popover
now also carries isNoteShellEditable: false, as the app's does). Glyph
shape and the Tier-2 fixed point are asserted only over an editable
host; ops/USJ idempotence and the Save write-back hold in all three.

New: Enter inside the popover's note content. It keeps the note whole
in every view, but over a hidden-marker host (formatted,
paragraph-structure) Enter is Lexical's own paragraph split: the caret
leaves the note for a new scaffolding paragraph, so the keystrokes that
follow are outside the note and the popover's Save (note op only) drops
them. Pinned as known failures.

knownFailureRunner now returns a registrar that appends the reason to
the test name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: run the corpus transform legs through the real Editor per view (PT-4197)

corpus-transform-fixed-point and the testUSFM transform leg mounted only
CharNodePlugin, MarkerEditPlugin and TextSpacingPlugin, so in formatted
and paragraph-structure the transforms those views actually run (the
note, paragraph and paragraph-marker-prefix plugins) never ran. Both now
mount the real Editor in each editing view through mountInView, dirty
every node once and read the document back with the adaptor Editor uses
on a local change. The composed provider-USX suite shares the same
runEveryTransform helper.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: annotation and undo rows in paragraph-structure and every editing view (PT-4197)

exactAnnotationLocations rows that stopped at standard/formatted (figure
caption hold, undo/redo, comment inside a caption, an NBSP inside a char
span, labels at a range's end) also run in paragraph-structure.
undoReachesFile's plain-text undo/redo control runs in all three
editing views.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: navigation-then-edit save pins in every editing view (PT-4197)

navigationThenEdit and scrRefNavigationThenEdit run in Standard,
formatted and paragraph-structure: an edit right after a navigation
must still reach onUsjChange (a lost onUsjChange is a lost save).

Each row starts from no document selection. Without that, a row that
follows another inherits jsdom's stale selection and stops detecting
the leak it pins: with a cursor-change tag forced onto the navigation's
caret placement, only every other row failed until the reset was added.
scrRefNavigationThenEdit also asserts the reported USJ carries the note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: host change-notification rows in every editing view (PT-4197)

onUsjChangeSettled's view-independent rows run in Standard, formatted
and paragraph-structure: once per content commit (a new plain-text
row; the attribute row stays Standard's), nothing for a selection-only
commit or a commit that changes no bytes (after a load, a reload, and a
paragraph touch after a lossless or double-space load), a local
applyUpdate announced once, and every remote applyUpdate row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: Editor undo and character-marker ref rows in every editing view (PT-4197)

The verse-spanning-delete undo regressions (guarded and native) and the
removeCharacterMarker/replaceCharacterMarker/extendCharacterMarker ref
rows that mounted the library's default (formatted) view now run in
Standard, formatted and paragraph-structure.

Two extendCharacterMarker rows fail in Standard view and are pinned as
known failures: coalescing two \bd runs leaves two adjacent \bd spans,
and extending \bd with \nd as a conflicting marker removes \nd but
never applies \bd while reporting success.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: note EditorRef suites in every editing view (PT-4197)

noteIndexAndHighlight, applyUpdateDomSelection and noteLooseTextApply
mount through optionsInView(view) in Standard, formatted and
paragraph-structure. The one row whose premise is Standard's display of
an unclosed note's caller (plain caller text, not an immutable caller
element) stays editable-marker only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: paragraph split and deletion content rows in every editing view (PT-4197)

charStackParagraphSplit and paraWholeDeletion pin the marker-edit
engine on hand-built Standard trees. Each gains real-Editor rows over
Standard, formatted and paragraph-structure judged on the saved USJ:
Enter inside \nd (and inside \wj \+nd) keeps both halves styled, and a
deletion of a paragraph's text, or across a paragraph boundary, keeps
the content the selection did not cover.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: word the navigation row-isolation comment as the hazard it is (PT-4197)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: satisfy the typecheck in two per-view rows (PT-4197)

The popover suite's known-failure map is a plain string map, and the
deletion rows' Delete keypress checks the editor root for null itself
(the marker-edit requireDefined admits null).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: never save an empty span's placeholder or drop an authored `~` (PT-4391)

Follow-up review fixes to the lone-NBSP content rule outside Standard view.

- CharNodePlugin: merging adjacent same-attribute glyph-less spans no
  longer carries an empty span's placeholder into the merged content.
  Landing after a nested span it read as content (the regression), and
  after a text-ending or `~` span Lexical merged it into the text before
  (`x~`, `~~`). The empty side now gives up its children and a caret on
  its placeholder moves to the seam. An author-marked `~` is never
  treated as a placeholder.
- Remove-style ($removeCharNodeKeepingContent) unwraps an author-marked
  lone `~` instead of deleting the span with it.
- NoteNodePlugin stripped a leading NBSP from any note char text on any
  transform (`\ft ~x` saved `\ft x`; typing after `\ft ~` lost it). It now
  uses CharNodePlugin's placeholder rule, which drops only a byte typed
  over a span that held nothing but the unmarked placeholder.
- $isCharContentEmpty counted any NBSP-only text as empty, so a
  close-and-reopen after a `~` span's `~` dropped the span with its `~`
  (unformatted, formatted, paragraph-structure). It now strips only the
  separator and asks $isLoneNbspContent about a lone NBSP.

Tests: gesture-level guards through the real Editor in every view
(lone-nbsp-gestures), CharNodePlugin merge/typing and NoteNodePlugin
units, $isCharContentEmpty units, and live/serialized agreement rows
inside spans and notes (read-only glyphs, collapsed-caller notes,
author-marked nodes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: preserve the USX/USJ version end to end (PT-4197)

A document now leaves the converters and the editor at the version it
arrived at, instead of being re-stamped 3.1 and forced back to 3.0 by the
host.

- `usxStringToUsj` keeps the root `<usx version>` (USJ_VERSION only when the
  USX declares none or an empty one); `usjToUsxString` writes `usj.version`
  (USX_VERSION only when the USJ has none).
- `Usj.version` is widened from `typeof USJ_VERSION` to `string` — a public
  type change, recorded in the utilities CHANGELOG and api report.
- The `usxVersion` option on `usjToUsxString` (Unreleased only) is removed;
  the round-trip audit suites now round-trip the version with no help, and
  the testUSFM USJ -> USX -> USJ net compares whole documents, version
  included.
- The editor remembers the version of `defaultUsj` / each `setUsj` and
  stamps it on every USJ it builds (load, local commit, apply, stale
  refresh, settled read via `$settledUsj`). The reverse adaptor takes the
  version to stamp; internal scratch USJ fed to the forward adaptor is
  unchanged. Loading a non-USJ_VERSION document no longer warns.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Swallow Enter inside an expanded note where markers are not editable

Outside Standard view nothing turns Enter in a note into a \fp break, so
Lexical split the paragraph holding the note and left the caret outside
it; what was typed next was lost on Save. NoteEnterGuardPlugin claims
KEY_ENTER, INSERT_PARAGRAPH and INSERT_LINE_BREAK at critical priority
while the selection touches an expanded note, in every markerMode but
"editable".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(platform-editor): save no space typed right after a char opener, as Paratext 9

Standard view saved a space typed directly after a char span's opener as
content (`\nd  things\nd*` -> content " things"). Paratext 9 skips all
whitespace after any opening marker (UsfmToken.Tokenize), so its Standard
view saves `\nd things\nd*`, and so does ParatextData on Platform.Bible's own
save path (ConvertUsxToUsfm runs NormalizeUsfm) — the editor's getUsj() was
the only place the space survived, so the editor reported a document the
file never held.

The span's content now drops its leading plain spaces on save, the rule a
paragraph's start and the text after a verse number already follow: the
screen keeps the typed space and the caret after it; the file does not get
it. The logical position model drops the same spaces so positions agree with
getUsj(). A span holding nothing but spaces keeps them (a selected space
wrapped in a span), since dropping that would join the words around it.

Catalog row O-5 (PT-4197). Pins updated deliberately: typedSeparatorSpace,
separatorGapSettle, verseAdjacentTyping, charAttributeTypedSettle (Standard
only; unformatted keeps whitespace), the adaptor caller-lookalike case, the
logical-model unit and oracle tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(platform-editor): report the space before a verse after a char span while pending

When a settle creates a verse directly after a char span (e.g. a `\` typed in
front of `v 2 b` after `\w grace\w*`), TextSpacingPlugin's verse transform
puts a space between them once the verse lands. The read-only settle behind
a pending getUsj() creates no live nodes, so it reported the span and the
verse touching, then the same bytes came back spaced after the settle.

The read-only settle now mirrors the transform over the output of every
paragraph or chapter scope it re-tokenizes. A scope the live settle only
renames in place (signature unchanged) re-creates no verse, so it is left
alone on both sides.

Paratext 9 has this space too: UsfmToken.Tokenize inserts one between a
closing marker and a following \v on every read, and the file is the same
with or without it (the writer puts a line break before \v) — measured with
ParatextData 9.5.0.24. Catalog row O-7 (PT-4197).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(platform-editor): rebuild committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep pasted prose inside the char span or note it is pasted into

Lexical inserts a paste's nodes (internal flavor, or text/html, which is what
Ctrl+V delivers) with RangeSelection.insertNodes, which splits every inline
ancestor of the caret up to the paragraph: text pasted inside \nd Lord\nd* split
the span and landed between the halves, and text pasted inside a note landed in
the paragraph after it (cut + paste back inside a note moved the text out).

A SELECTION_INSERT_CLIPBOARD_NODES_COMMAND handler now inserts a payload of plain
prose as text when the paste starts in ordinary text inside an inline element,
the way typing it would. Structured payloads are still Lexical's.

Found by the PT-4197 clipboard corpus sweep (finding CB-2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Keep the no-break space a cut leaves as a char span's only text

In the views without char glyphs a span's lone NBSP is also the empty-span caret
placeholder, which the save drops unless the text is tagged as the author's. Only
the USJ-to-editor adaptor tagged it, so cutting or deleting `trail` from
`\nd trail~\nd*` saved an empty span: the NBSP outside the selection was lost.

A CharNodePlugin transform now tags a lone-NBSP text in a glyph-less span as
authored when that text held more than the placeholder in the last committed
state.

Found by the PT-4197 clipboard corpus sweep (finding CB-4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Sweep copy, cut and paste over the testUSFM corpus in every editing view

Standard view's USFM clipboard has a corpus sweep; formatted and
paragraph-structure had none. The new suite samples whole verses, ranges
starting or ending inside a char span, ranges holding a footnote, crossing a
paragraph boundary or a verse number, spanning a paragraph, and inside note
content, then copies and pastes (all flavors, html+plain as Ctrl+V delivers,
plain), cuts, cuts and pastes back, and pastes USFM-looking text, checking each
result and its undo/redo against a USJ-level model.

Pinned with knownFailureRunner (68 rows):
- paragraph-structure: a copy across a paragraph boundary carries the gutter
  glyph (\q1) in text/plain, so CommandMenuPlugin refuses its paste (CB-1);
- hidden views, html flavor: char attributes (closed) are not carried and a
  note's display spacers come back as text (CB-3);
- Standard reference rows over damaged USFM (2SA-5) and one derived-sid loss.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rebuild the committed dist for the clipboard fixes

Carries the inline-prose paste and the remaining-NBSP transform into
packages/platform/dist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: typing-fidelity oracle harness for the hidden-marker views (PT-4197)

A keystroke oracle for formatted and paragraph-structure, where no
pending marker edit exists to settle. The expectation for each keystroke
is computed from the loaded USJ alone: each leaf block read as units (a
character, or an atom shown whole: verse, collapsed note, milestone,
figure), a caret as a unit index, and typing, Backspace, Delete, Enter
and typing over a range edited into the USJ tree there. The saved USJ
must be one of the expected documents, Undo must restore the loaded USJ
exactly and Redo re-apply the keystroke's.

Keystrokes arrive as a browser delivers them: the caret placed with a
selection change, the DOM selection made the editor's, a real keydown
on the root, then beforeinput and the text insertion. Selection.modify
(which jsdom lacks and Lexical's deleteCharacter needs) is supplied at
character granularity. Failures are reported and listed by shape.

Exports the pending-equals-settled oracle's keystroke corpus for reuse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: Delete before a collapsed note no longer throws (PT-4197)

In the hidden-marker views, Delete just before a collapsed note removes
its caller; NoteNodePlugin then removed the note and went on to move its
first child out of the removed node, which threw ('Expected node to have
a parent'), discarding the edit. Return once the note is removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: joining a paragraph into an empty \b keeps its text (PT-4197)

ParaNodePlugin refuses text typed into an empty blank line by clearing
it. A join with the paragraph after it (Backspace at that paragraph's
start, Delete in the blank line, or a range across the break typed over)
also lands content in the empty \b, and the clear deleted that whole
paragraph's text in the formatted view. Clear only when the paragraph
after the blank line is still there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: Undo takes back Enter in front of a char span or a note (PT-4197)

Lexical's history folds an update whose only dirty leaf is a text that
kept its text into the previous undo step, even when tagged as a push.
A paragraph break in front of an inline element is such an update in the
formatted view, so Undo did not take the break back. ParaNodePlugin now
marks a created paragraph's first leaf dirty as well.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: an author's no-break space left alone in a span is saved (PT-4197)

In the views without a span separator (hidden and visible markers), a
lone NBSP in a span is the empty-span placeholder unless it carries
authoredNbspState, which the adaptor set only on a span text that was
exactly one NBSP at load. Deleting the rest of \nd ~x\nd* (or x~)
left an unmarked NBSP, and the span saved empty; an NBSP typed into an
empty span lost to the placeholder drop the same way.

The adaptor now marks every span text holding an NBSP (Lexical keeps
node state through text edits and copies it on splitText), and the
placeholder drop marks the NBSP it leaves behind.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix: typing over a range across a span edge or a note keeps the rest (PT-4197)

Lexical's RangeSelection.insertText moves the content after a range's
last text to after its first text, into the inline element the range
starts in, and resolves a range start at a text's end into the inline
element after it. In the hidden-marker views that pulled the text after
an enclosing char span inside the span (out of order), and, for a range
starting just before a collapsed note, put the typed text and the rest
of the range into the note, which was then removed with them.

InlineRangeEditPlugin (mounted for markerMode hidden) replaces a range
over more than one text of one paragraph itself: it cuts the edge texts,
removes the nodes between and the spans and notes that leaves empty,
and puts a point Lexical resolved into a collapsed note back beside it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: run the typing-fidelity oracle in formatted and paragraph-structure (PT-4197)

Two files per view (the keystroke corpus with the short testUSFM
chapters; the long testUSFM and WEB chapters), paragraph-structure under
Simple mode's guarded structure protection. The shapes still failing are
listed with their reasons: Redo lost or a second Undo needed around an
empty verse, one Enter-at-paragraph-start Undo in WEB Matthew 5, and one
ignored Backspace at an element caret in 1 Samuel 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build: rebuild the committed platform output (PT-4197)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(platform-editor): rebuild committed dist for the typing and clipboard merges

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* test(platform-editor): hold each keystroke to keeping content, and pin separator removal

The pending/settled oracle compared the pending document with the settled
one, so content a transform dropped in the keystroke's own update was
invisible: the pending document already lacked it and settled to itself.
A mutation that removes a span's following text once its separator is
deleted passed every check. Each edit's screen is now read as the edit
left it (before transforms) and as it landed, and the landed screen may
only respell the spliced one: no content byte (whitespace and marker
tokens aside) may go. Range keystrokes are excluded: the oracle replaces
a range with a direct insertText that skips the commands the marker-edit
engine answers, so their transforms are not what a keystroke does. All
four oracle views stay green with nothing new listed.

Catalog row O-8 (PT-4197) reported Backspace after a `\w` opener's
separator losing the span and the text after it. It does not: the bytes
on screen (`\p In the \wgrace|grace\w* of God`) settle, pending and
settled alike, to `\p In the ` and a new unknown paragraph `\wgrace`
holding the rest — ParatextData's own reading (an unknown marker is a
paragraph outside a note, a char span inside one; checked against its
USX). The report came from a probe that printed only the first block.
New cases pin it for Backspace and Delete across attributes, `\nd`,
nesting and a note, comparing the whole document with ParatextData's USX.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix(platform-editor): save a char span holding only spaces as empty, as Paratext 9

Standard view already dropped the spaces after a char opener's separator,
but kept them when they were all the span held (a selected space wrapped
in `\nd`). ParatextData skips all whitespace after an opener there too:
`one\nd  \nd*two` saves as `one\nd \nd*two` (USX `<char style="nd" />`),
so the words join in the file. The editor's saved USJ now matches; the
screen still shows the space inside the span. The logical position model
drops the exception with it, so positions agree with getUsj().

A no-break space is data and stays: `\nd ~lead\nd*` and `\nd ~\nd*` keep
it on load and with a space typed in front of it (new tests; ParatextData
keeps both too).

Pin updated deliberately: markerMenuApply's whitespace-only wrap now saves
`["one", {nd}, "two"]` while the live span still holds the space.

Catalog row O-5 follow-up (PT-4197).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform-editor): rebuild committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix(shared-react): keep alike char spans apart once a deletion leaves them side by side

In the hidden-marker views (formatted, paragraph-structure) CharNodePlugin
combined two adjacent same-marker, same-attribute spans whenever either was
dirtied, so deleting the text between `\w one|lemma="x"\w*` and
`\w two|lemma="x"\w*`, or joining the paragraphs they end and start, saved one
`\w onetwo\w*` and changed word segmentation (typing oracle finding T-7).
Standard view keeps them apart (its spans carry glyphs), and so does
Paratext 9.

Two spans that were already alike, separate spans in the last committed state
now stay separate. Spans an update creates (a marker applied to a run, a
split's continuation, remote content) still combine with an alike neighbor,
as does a span whose marker an update changes to match.

- Regression tests in every editing view plus paragraph-structure (guarded):
  deleting the space between, and joining the paragraphs.
- CharNodePlugin unit tests that expect a combine now add their spans by an
  update once the plugin runs (they previously relied on the plugin's
  registration re-dirtying the initial state, i.e. spans the document already
  held); new unit test for the keep-apart case.
- Typing oracle: drop the "adjacent alike spans are one" allowance; both views'
  corpus and chapters pass without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform-editor): rebuild committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* test(clipboard): name the tickets that fix the clipboard corpus pins

The paragraph-structure gutter-glyph copy pins (CB-1, 44 rows) name
PT-4911, and the hidden-view html-flavor pins (CB-3, 13 rows) name
PT-4921, in a TODO beside each rule and at the head of its reason. The
Standard-view damaged-USFM and derived-sid rows have no ticket yet and
are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix(shared-react): arm a chapter for the guarded two-press delete

Under "guarded" structure protection (Simple mode) a chapter marker beside the
caret was outside the two-press guard: Delete at the end of the block before
`\c N` removed the chapter on the first press with no arming, and Backspace at
the start of the paragraph after it armed that paragraph, whose merge then
did nothing because a paragraph cannot merge into a chapter (typing oracle
finding T-10).

$structuralDeleteTarget now resolves the atomic chapter glyph
(ImmutableChapterNode) beside a block edge as a "chapter" target. The first
press node-selects it (as for a verse) and the armed chapter carries the
verse-selected class, so it shows the armed styling; the second press removes
it and returns the caret where the first press found it. The opposite key
cancels. No host hint kind is published for a chapter (the host's hint copy is
verse/selection-only), matching paragraph merges. Standard view (editable
chapter text), "protected" (still refused) and "off" (Power mode) are
unchanged.

Tests: guardedChapterDeletion.test.tsx in formatted and paragraph-structure
views - Backspace after / Delete before `\c 1` and `\c 2` arm then delete
(guarded), cancel on the opposite key, refused under protected, native under
off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix(platform-editor): keep a nameless character marker's empty name on export

ParatextData reads a bare `\` inside a note (`\f + \ft \ b\f*`) as
`<char style="" status="unknown">`. The converters already keep the empty
style as `marker: ""`, but the editor's export turned an empty char marker
into no marker at all, so getUsj() handed back a char with no name and the
USX written from it was not the USX loaded. createCharMarker now keeps
`marker: ""`, the way nameless paragraphs already round-trip.

A char with no marker at all in its USJ (not a shape USX can express) now
exports as `marker: ""`, as an absent paragraph marker already exports as `p`.

Tests: namelessChar.test.tsx loads ParatextData's captured USX for an unclosed
and a `\*`-closed nameless span in a footnote, and asserts getUsj() and the
USX written from it are unchanged in Standard, formatted and
paragraph-structure views; converter tests for a char's empty style.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* docs: correct what ParatextData does with a marker-bearing \cp

Comments claimed ParatextData keeps every `\cp` that holds a marker as its own
marker. Measured, it folds the text before the first marker into the
chapter's pubnumber and leaves the rest as loose content after the chapter;
only a `\cp` whose content starts with a marker stays a `\cp`. The editor
deliberately settles such a `\cp` to the USFM-spec shape: the saved bytes are
the ones Paratext 9 writes, and a reload shows Paratext 9's shape. No
behavior change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform-editor): rebuild committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* feat(shared-react): publish a chapter hint kind for the guarded delete

An armed chapter in Simple mode's two-press delete now sets
data-verse-delete-kind="chapter" (with data-verse-delete-intent) on the
editor root, as an armed verse or selection does, so the host can show a
chapter-specific "press again" hint. A paragraph merge still publishes no
hint kind. CHANGELOG records the chapter arming and the new value.

Tests: guardedChapterDeletion (formatted + paragraph-structure) and the
StructureKeyboardPlugin DOM-signal suite expect kind=chapter.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* test(platform): Standard view may drop an empty derived sid on a verse paste

Pasting `\v 9 x` as plain text in Standard view rebuilds part of the
paragraph it lands in, and the 2SA-3 fixture's empty derived `sid`s on the
re-derived verses do not survive. Dropping derived metadata is accepted
(data-fidelity catalog F-8), so this is now an expectation rather than a
known failure: in the paragraph the verse lands in, an empty `sid` may be
dropped, and everything else (non-empty `sid`s included) must still match
exactly. The row still performs the paste, undo and redo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* test: name the tickets for the remaining untracked expected failures

- PT-4924: typing-oracle T-11 / T-12 / T-16 entries in both views'
  expected-failures JSON (in each reason).
- PT-4925: Standard view's extendCharacterMarker coalesce/conflict rows
  (EXTEND_KNOWN_FAILURES).
- PT-4388: the Standard-view 2SA-5 damaged-USFM clipboard rows and the
  fragment tokenizer's "testUSFM 2SA 5" known divergence (bare `\` as a
  nameless marker, waiting on ParatextData 9.6).

Every pin and what it asserts is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix: a join into an empty \b keeps the joined paragraph, as Paratext 9 (PT-4197)

Backspace at the start of a paragraph after an empty \b, Delete in the
blank line, or a range across the break typed over joined the paragraph
into the \b and kept the blank-line marker on text. Paratext 9's
formatted view joins paragraphs the way Gecko joins two blocks: the
second block survives (JoinNodeTransaction keeps the right node), so the
blank line goes and the paragraph keeps its own marker. ParaNodePlugin
now gives a blank \b that takes in a joined paragraph that paragraph's
marker and attributes, rewriting a gutter glyph to match, in formatted,
paragraph-structure and Simple's guarded two-press join. Standard view
edits marker text and is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* test: a typed U+00A0 saves as a space in Standard view, as Paratext 9 (PT-4197)

Paratext 9's Standard view (AllowInvisibleChars off) saves every U+00A0
in the editor as a space (UsfmHtmlUtilities.LegacyPostprocessUsfm) and
the author's no-break space is typed as `~`. Ours does the same; pin it
between letters, after a space and at a span's start, and pin `~` typed
saving an NBSP.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build: rebuild committed dist for the \b join fix (PT-4197)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix(shared-react): paragraph-structure Enter drops the break's space

Enter at a word break (`In| the`) left ` the` in the new paragraph in the
paragraph-structure (Simple) view, while Standard and formatted save
`the`. ParaNodePlugin trims a new paragraph's leading space from its first
child, which in that view is the gutter marker glyph rather than the text,
so nothing was trimmed. It now looks past a leading gutter glyph.

Tests: typingFidelity.regressions "Enter at a word break" in all three
editing views, unguarded and guarded (red in paragraph-structure: saved
" the beginning"). The typing oracle no longer accepts a new paragraph
that starts with text keeping its leading space (it fails 23 corpus rows
without the fix).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform): rebuild the committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* feat(platform): ViewOptions.allowInvisibleCharacters keeps ~ and U+00A0 apart in Standard view

A Paratext project with AllowInvisibleChars on stores a no-break space as
U+00A0 itself and `~` as an ordinary character (measured with ParatextData:
load, save and round trip all carry both verbatim and distinct). Standard
view always applied the `~` = NBSP rules, so a literal `~` in such a
project was read as a no-break space and saved as U+00A0, and a U+00A0
typed there saved as a space.

With the new option on, a stored U+00A0 displays as NBSP_SYMBOL (U+237D)
instead of `~`, `~` stays an ordinary character, and every layer that
maps display or fragment text to data takes the spelling from
getNbspSpelling(viewOptions): the load/save adaptors, the tier-2 and
virtual settle tokenizer (new UsfmFragmentOptions.nbspSpelling), the
Unformatted content fragments, paste, copy (U+00A0 written as itself),
the Markers view's copy, and a typed U+00A0 (command and native
beforeinput routes). getViewMode ignores the field. Off is unchanged.

Tests: invisibleCharacters.test.tsx (load/save, edit, span and note,
typed ~, typed U+00A0 by both routes, paste, copy, settle and unsettled
read, Unformatted settle, Markers copy, formatted/paragraph-structure
pins), tokenizer nbspSpelling, getViewMode/getNbspSpelling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform): rebuild the committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* feat(platform): toUsfmNbspSpelling for hosts that read the editor's display text

A host that copies or searches text read from the editor's DOM (the
footnote popover's Copy button, a Find seeded from the selection) sees
Standard view's display spelling of a no-break space. With
allowInvisibleCharacters on that is U+237D, which is not the project's
text; this turns it back into U+00A0, and leaves the text unchanged when
the option is off (`~` is the project's spelling then).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform): rebuild the committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* test(platform-editor): keep a note's empty caller empty through edits in every view

ParatextData reads `\f \ft x\f*` (one or two spaces after the marker) as
caller="" and writes it back as `\f \ft x\f*` (measured with the corpus
capture harness), and the tokenizer has followed that since PR #11. The
existing rows covered only the expanded editable views; this pins a
load, an edit beside the note and one inside it, and a departure in
every editor view, saving caller "".

Fails in all 8 views if the exporter writes "+" for an empty caller.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* feat(platform): U+001A is the no-break-space stand-in, drawn as a blank, and never leaves the editor

With ViewOptions.allowInvisibleCharacters on, Standard view held a stored
U+00A0 as U+237D (shown as a visible box glyph). U+237D can occur in
Paratext 9 text, so a literal one was saved as a no-break space. It now
holds U+001A (SUBSTITUTE), which ParatextData's ScrText.GetText turns
into a space on every read, so no project text can hold one.

- Rendering: a one-glyph font (empty, 0.25em wide, unicode-range U+1A,
  NBSP_STAND_IN_FONT_FAMILY) draws it as a blank the width of a space.
  The editor adds the face to its document through the CSS Font Loading
  API (from bytes, so no CSP font-src is involved). Chromium draws a
  control character as a box unless the FIRST family has a glyph for
  it, so generateUsjCss and usj-nodes.css lead every Scripture
  font-family with it.
- It never leaves the editor: delta ops given out (onUsjChange,
  getNoteOps, getOpsAfterNote, takeOpsAfterNote) carry U+00A0 for it and
  a space for an NBSP the view shows in a space run; applyUpdate and
  replaceEmbedUpdate take a U+00A0 that is not a marker separator as a
  stored no-break space; the internal Lexical clipboard flavor is left
  out when the selection holds a stand-in; a drag out carries the copy's
  USFM text and html; annotation callbacks and note-caller previews give
  U+00A0.
- Incoming text: every C0 control character but tab, LF and CR becomes a
  space in Standard view before anything reads it (load incl. attribute
  values, paste, drop, typing by both routes, applied ops).
- A drop into Standard view now goes in as a paste of the same data.
- The fragment tokenizer keeps the spelling as a character
  (regularizeSpaces turned control characters into spaces first).

Leaking before (measured against the U+237D build): ops out, note ops,
the footnote-editor round trip, ops in (U+00A0 saved as a space), the
Lexical clipboard flavor, the cut, the async paste, drag and drop, and
annotation text. Forced-on run (every registry view with the option):
only the 5 intended off-mode pins fail.

Known gap (both settings, not changed here): an attribute value's
U+00A0 is shown raw in its attribute run, and a settle of its paragraph
reads it as a space.

Tests: nbspStandIn.test.tsx (35), whitespaceDisplay.utils.test.ts
(control characters, ops mapping), usj-nodes.css and generateUsjCss
font stack, invisibleCharacters.test.tsx on the new stand-in.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix(platform): a note caller's click hands out the note's ops with U+00A0

The getNoteOps a noteCallerOnClick handler receives read the note's ops
straight from the tree, so with allowInvisibleCharacters on they carried
the internal stand-in; Platform.Bible's footnote popover applied them,
and the incoming rule turned each stand-in into a space (seen in the
running app). They now go out through opsForHost like every other op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform): rebuild the committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* Address review-paratext findings on the data-fidelity branch

- Fix InlineRangeEditPlugin: typing over a range whose two edge texts are
  both wholly selected no longer falls through to Lexical, which pulled the
  text after a char span into the span. The plugin now steps aside only for
  a whole-edge deletion. Regression rows added to
  typingFidelity.regressions.test.tsx.
- Add standardViewDrop.test.tsx: drop-as-paste rows under both
  allowInvisibleCharacters settings.
- Add the missing CHANGELOG entries (platform Fixed section; utilities note
  that the Scribe editor does not yet carry the USJ version through).
- Replace in-code Jira/branch references with forward-facing wording per
  CLAUDE.md, keeping open TODO(PT-x) markers.
- Document that allowInvisibleCharacters must be right before the first
  load; correct the vertical-tab/form-feed wording; clipboard-semantics
  doc updates.
- Cleanups: $collapsedNoteOf uses $findMatchingParent, ParaNodePlugin key
  lookup, CharNodePlugin TSDoc, un-export $isSelectionInExpandedNote,
  NoteEnterGuardPlugin ordering comment, corpus fixed-point test uses
  knownFailureRunner, reshaped half-flat test trees, comment re-wraps,
  demo armed-chapter CSS.
- Rebuild committed packages/platform/dist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-URL: https://claude.ai/code/session_01KiNAWjh4KAwjzn1g34EePL

* fix(platform): an attribute value's no-break space keeps its spelling in Standard view

Standard view showed a U+00A0 inside an attribute value as a raw U+00A0
in its attribute run, which the engine reads as display spacing, so a
settle of the paragraph (Tier-2 rebuild or virtual settle) saved it as a
plain space, in both ViewOptions.allowInvisibleCharacters settings. The
value now takes the body text's spelling (`~`, or the U+001A stand-in
with the setting on) everywhere attribute state becomes display text:

- the adaptor's runs for char spans, milestones, \va/\vp, \cat, \ca/\cp
  and an UnknownNode's display parts (figure src, sidebar \cat, periph);
- the display-run syncs, which redraw from node state without view
  options: MarkerEditPlugin registers the editor's spelling
  (registerNbspDisplaySpelling / $getNbspDisplaySpelling in shared);
- the `|...` bytes an unwrapped or emptied span leaves as content, a
  collapsed note's copied \cat, and the annotation byte maps.

The tokenizer reads the spelling back as U+00A0 inside milestone
(including an absorbed list), figure and \periph attribute values, as it
already did for body text, char-span attributes and the attribute
markers. Unformatted view, which shows a value's U+00A0 as itself, now
respells it past the run's leading separator in its settle fragment
(attributeFragmentText), as it does content, instead of flattening it to
a space.

Tests: attributeValueNbsp.test.tsx (each producer shape in Standard view
with the option off and on: display, paragraph settle, value edit read
before and after the settle, copy and paste with every flavor and with
plain text alone; figure; span unwrap and paragraph-break emptying; a
collapsed note's copied \cat; Unformatted; formatted and
paragraph-structure pins) and tokenizer cases in
usfmFragmentToUsj.test.ts. The nbspStandIn chapter row now settles the
chapter line instead of pinning the raw display.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* build(platform): rebuild the committed dist

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RDpA6DiFYGheLpPSaj6yPz

* fix(shared-react): rejoin a char span that Enter split when Backspace joins the paragraphs back

Keeping alike char spans apart once both are in the document also kept apart
the two halves of one span that Enter cut in two, so Enter then Backspace
inside `\nd thing\nd*` saved `\nd th\nd*\nd ing\nd*` in the views that hide
markers.

- CharNode.insertNewAfter records, per editor, which span each split
  continuation came from; $wereSplitFromOneSpan lets those pieces merge again.
  Spans the document held apart still stay apart.
- Regression test in every hidden-marker view: Enter inside a span, then
  Backspace, saves one span.
- InlineRangeEditPlugin: comment the composition gap (an input method started
  over a cross-span range takes Lexical's own path).
- CHANGELOG entry extended.
- Rebuild committed packages/platform/dist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-URL: https://claude.ai/code/session_018WCZ5KDNh1Vqzt3JP6jCnp

* fix(shared-react): keep Undo/Redo working around empty paragraphs and caret hosts

- Undo after a guarded two-step removal of an empty paragraph (an empty
  `\b`) now brings it back. Lexical's history folds an update whose only
  dirty leaf is a text that kept its text into the step before, even one
  tagged history-push, so `$mergeParaIntoPrevious` marks the removed
  paragraph's leaves dirty to make the removal its own undo step.
- Redo works after an Undo leaves the caret beside an empty verse or a
  trailing note. Caret-host commits (create, move, removal on blur) are
  tagged history-merge through `$tagCaretHostCommit`, so they no longer
  become an undo step that clears the redo stack.
- Drop the testUSFM-2SA-2/2SA-3 rows from both typing-oracle
  expected-failure lists: the Undo/Redo shapes they pinned now pass.
- Pin an author's literal `~` in a milestone attribute value turning into
  U+00A0 on a Standard-view settle with `it.fails` (needs a product call).
- Use a code span for `USJ_VERSION` in `Usj.version`'s TSDoc so
  consumers' typedoc can resolve the comment.
- Rebuild the committed dist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Session-URL: https://claude.ai/code/session_018WCZ5GnNin2H8EzQyWWq6p

* fix(platform): keep a pasted no-break space at the start of a line

Standard view's paste read a U+00A0 at the start of the pasted text, or
of any line of it, as display spacing and inserted a plain space. So a
pasted lone no-break space became a space, one beside a space collapsed
away, and with allowInvisibleCharacters on the editor's own copy of a
stored no-break space pasted back as a space or as nothing (the copy
spells a stored no-break space as U+00A0 there).

No copy reaching the paste puts display spacing at a line start: the
editor's own copy writes a plain space for every display NBSP in both
readable flavors, a browser's default copy writes text/plain with every
NBSP as a space, and a Paratext 9 clipboard is decoded with its NBSPs
resolved. normalizePastedNbsp now keeps every pasted U+00A0 as a stored
no-break space except one right beside a marker token, which is still
read as a separator.

pastedNbsp.test.tsx pastes each case through the editor in both
settings: from a foreign text/plain and html-only clipboard, and the
editor's own copy pasted back. Two rows stay expected failures with the
setting on: a stored no-break space right after a closer or right
before an opener copies as U+00A0 and pastes back as a space under the
marker-adjacent rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(platform): stop exporting NBSP_STAND_IN_FONT_FAMILY

Hosts name the no-break-space stand-in's font family in their own
stylesheets, which cannot import a constant, so the export gave them
nothing to use. The constant stays internal to generateUsjCss and the
font registration, and the docs now state the family name,
PlatformEditorNbspStandIn, which must lead any host Scripture
font-family and must not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): keep an author's lone no-break space in delta ops

- The delta-op producer dropped every text that is exactly one U+00A0
  inside a note's contents, and every char span whose only text is one
  U+00A0 (as an empty span's placeholder). In views that hold a stored
  no-break space as U+00A0 itself (formatted, paragraph-structure), that
  is also a `~` between two of a note's char spans, or a `~` that is a
  span's whole text.
- Both checks now ask `$isLoneNbspContent`, the rule the editor-to-USJ
  conversion already keeps content by, so a note's ops (`getNoteOps`,
  which Simple mode's footnotes pane edits) carry what `getUsj()` saves.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(platform): Standard view's delta ops read as USJ text in both no-break-space settings

- `opsForHost`/`opsFromHost` told a marker's separator from a stored
  no-break space only within one op, and took any op that is exactly one
  U+00A0 as a spacer. The editor emits each text node as its own op, so a
  glyph's separator (`\nd`, then ` A`) went out as a space, and with
  `allowInvisibleCharacters` on a stored no-break space that is a whole
  op's text (a `\ft` that is only `~`, a `~` between two spans or after a
  note, text carried out past a note) came back in as display spacing and
  was saved as a space or dropped. An NBSP is now structural only right
  after an opening glyph, in the same op or ending the op before.
- With `allowInvisibleCharacters` off, Standard view's ops went out as its
  display text: `~` for a stored no-break space and U+00A0 for each space
  of a run. Simple mode's footnotes pane edits a paragraph-structure
  editor's notes in Standard view, so a pane edit saved a space run as
  no-break spaces, and a stored no-break space showed in the pane as a
  blank rather than `~`. Both settings now map the same way: the stored
  no-break space's spelling to U+00A0 and a run's NBSPs to spaces going
  out, U+00A0 to the spelling coming in.
- A paragraph-structure editor still reads a char span whose whole text
  is one U+00A0 from applied ops as an empty span's placeholder, so the
  pane drops that `~`; pinned with `it.fails`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(platform): mark toUsfmNbspSpelling experimental

It stays exported, but is expected to be removed once Standard view
holds a stored no-break space as itself (PT-4931): display text will
then already carry the project's spelling.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): keep an applied no-break space that is a char span's whole text

- Applying delta ops built every char span's text without the mark a
  load puts on a span's no-break space (`authoredNbspState`). In the
  marker modes without a separator after a span's opener (formatted,
  paragraph-structure), a span whose whole text is one U+00A0 then read
  as an empty span's placeholder and saved as nothing: Simple mode's
  footnotes pane dropped a `\ft ~` (or any `~`-only span) its row editor
  handed back, and the same happened to a `~`-only span inserted by
  `applyUpdate`.
- The op producer leaves an empty span's placeholder out of its ops, so
  an applied U+00A0 is always content. `$markAppliedNbsp` marks the text
  of every char-span insert (`$handleCharText`, a note's contents, an
  unknown embed's contents) that holds one, as the USJ adaptor marks a
  loaded span's text; editable marker mode is left unmarked, as on a
  load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(platform): expect the footnotes pane to keep a `~`-only char span

The paragraph-structure main editor now marks an applied lone no-break
space as content, so the row that pinned the drop with `it.fails` runs
as an ordinary test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): keep a char span or milestone in text carried out past a note

Text a host's note editor carries out past its note (`takeOpsAfterNote`,
applied after the note with `replaceEmbedUpdate`) lost its markup:

- A char span from a Standard-view note editor became an unclosed span
  plus a stray `\nd*` in Standard view, and its glyphs became span text
  in the paragraph-structure and formatted views. The carried ops held
  the span's marker glyphs as text, the shape of the note editor's own
  delta document. `getOpsAfterNote`/`takeOpsAfterNote` now produce
  content-only ops (`$getParticularNodeOps`'s new `contentOnly` option),
  the shape a note's contents already have, and `takeOpsAfterNote`
  deletes what it carries by the nodes' apply-coordinates length
  (`$getApplyLength`), since the ops no longer count the glyphs.
- Applying a char span in editable marker mode added its glyphs but
  advanced past its text alone, so the ops after it landed inside the
  span's opener. `$handleCharText` now gives a fresh span the separator
  after its opener, as a load does, and counts everything it added.
- Plain text inserted at the end of a nested span went into the outer
  span; it now goes after the outermost span.
- A milestone vanished in every view: the op producer writes a milestone
  under `milestone`, and the applier read only `ms`. It now reads both.

Standard view still splits a carried `\nd L \+wj M\+wj*\nd*` into two
`\nd` spans, pinned with `it.fails` in noteCarriedText.test.tsx.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(platform): pin copying a whole note through the editor's own copy

A host's copy-this-note button selects the whole note, calls
EditorRef.copy() and puts the selection back with setSelection(). Pin
that the sequence writes the USFM a Ctrl+C of the note writes (no
display NBSP, a stored NBSP as ~ or U+00A0 by setting, no closer an
unclosed note lacks, no text after the note), applies the copy limit
(a note that does not fit copies nothing), and leaves the selection,
the document and a pending marker edit as they were when it runs in
one synchronous handler. The mount helper takes a copy limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(platform): read a pasted no-break space as a space only after an opening marker

A paste turned a U+00A0 right after any marker token, or right before
one, into a space. With allowInvisibleCharacters on, Standard view's own
copy writes a stored no-break space as U+00A0, so copying one that sat
after a closer or in front of a marker and pasting it back saved a
space.

Only a U+00A0 directly after an opening marker (\nd, \+nd) is now read
as a space: the marker's separator, as ParatextData reads it. After a
closer or a milestone's \* self-closer, and in front of any marker, a
U+00A0 is kept as a stored no-break space. A foreign copy of the display
that carries a run's or a note layout's spacing there (the⍽\nd,
\f +⍽\fr) now keeps it as a no-break space rather than a space: nothing
is lost. The two own-copy round-trip rows that pinned the old reading as
expected failures now pass; the paste suites are re-pinned to the new
reading, and a pasted \nd~Lord\nd* is pinned to keep every byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): emit milestone ops under the ms key the op types declare

`$getMilestoneOp` wrote a milestone as `{ insert: { milestone: … } }`, but
`OTEmbedTypes`/`validOTEmbedTypes` declare `ms`, and so do
`isInsertEmbedOpOfType` and the applier. A host reading ops through the
public type guard never saw a milestone, and `getInsertedNodeKey` (which
only recognises `validOTEmbedTypes`) could not resolve one. The producer
now writes `ms`; the applier keeps reading `milestone` too, so ops from
earlier builds still apply their milestones.

Tests that pinned `milestone` (adaptor ops, the shared converter
fixtures) now expect `ms`, the applier's `milestone` test is named as the
legacy-key case, and a produce-then-apply round trip pins the producer
key and that both milestones survive with sid/eid/who. Before the
producer change the round trip's apply half already passed (4e43000f
made the applier read both keys); the key pin failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): build adjacent char ops of different attributes as separate spans

Applying ops joined a char op into the span before it, or into the span
at the insertion point, when the two matched on marker and cid alone. So
the footnote editor's round trip (getNoteOps, applyUpdate,
replaceEmbedUpdate) turned `\w a|lemma="x"\w*\w b|lemma="y"\w*` into
`\w ab|lemma="x"\w*` in every view, and an op inserted right before a
`\w` span of another lemma went into that span's text, losing its own
lemma (and, for a nested char array, its outer span).

`$hasAllSameCharAttributes` (shared) compares the marker, the cid and
every other attribute. `$createNestedChars` and `$handleCharText` use it
to decide whether an op continues an existing span, and
`$charNodeTransform` uses it in place of its own attribute comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): keep adjacent spans that arrive in one paste apart

In the views that hide marker glyphs, `$charNodeTransform` joined two
alike spans that a paste inserted side by side, so pasting the editor's
own copy of `\w a\w*\w b\w*` or `\nd A\nd*\nd B\nd*` saved one span.
Standard view, and Paratext, keep them as the two spans the copied
content held.

`CharNodePlugin` records the char spans each clipboard insertion brings
in (`SELECTION_INSERT_CLIPBOARD_NODES_COMMAND`, and the protected-document
paste and drop, which insert directly), and `$charNodeTransform` keeps
two of them apart for the rest of that update. A pasted span still joins
an alike span the document held, so pasting a piece of a span inside a
span of the same marker extends it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(platform): rebuild the committed dist for the merged paste and ops fixes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): count an author's `~`-only char span in delta-doc positions

- The ops stream emits a char span whose whole text is an author's
  no-break space (`$isLoneNbspContent`), but delta-doc counting still
  took every lone U+00A0 that is a span's only child as the empty span's
  placeholder and counted it 0. Every delta-doc position after such a
  span (`\nd ~\nd*` in formatted, paragraph-structure and the other views
  without a separator after the opener) was one short of the ops stream,
  so a fast-path keystroke after it went out with a retain one too small.
- The placeholder check now asks `$isLoneNbspContent`, as the producer
  does. Apply coordinates are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): keep a note's `~` that follows the opener's separator in delta ops

- In editable marker mode (unformatted) the text after a note span's
  opener is the separator followed by the author's text. For `\ft ~\ft*`
  or `\fk ~\+wj x\+wj*\fk*` that is two no-break spaces; the producer
  stripped the separator and then asked `$isLoneNbspContent` about the
  node, whose whole text is not one U+00A0, so the author's `~` was taken
  for layout and dropped from the note's ops. `getNoteOps` and
  `getEditorDelta` then disagreed with `getUsj()`, which keeps it.
- A lone NBSP left after stripping a separator is now content; the
  position rule is asked only about text that kept its node's every byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): give an applied `~` the separator after a span's opener in editable mode

- `$createNestedChars` told an empty span's placeholder from first
  content by its bytes, so an op whose text is one U+00A0 (an author's
  `~` as a span's whole text, or leading a nested span) got no separator
  after the opener in editable marker mode. Saving then stripped the `~`
  as the separator: in the unformatted view, the footnote popover's save
  and the Simple-mode pane round trip both lost `\ft ~\ft*` and
  `\fk ~\+wj x\+wj*\fk*`.
- The placeholder is now known by where it came from (an empty op's
  text), carried through the merge recursion; every op's own text takes
  the separator. An empty op still gets only the placeholder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(platform): pin that saved USJ and delta ops agree on every lone `~`

- `lone-nbsp-agreement.test.tsx` runs each lone-`~` shape (a span's whole
  text, between two spans, leading an element-first span; in body text
  and in a note) through every view in both settings of
  `allowInvisibleCharacters`: the saved USJ keeps it, `getEditorDelta`
  carries the same lone NBSPs, delta-doc positions after it match the
  ops stream, and a note's `~` survives a save of its own ops, the
  footnote popover's round trip and the footnotes pane's round trip.
  Text typed into an element-first spacer or an empty span's
  placeholder saves as content.
- The popover round-trip suite gains an authored `\fq ~\fq*` and
  `\fk ~\+wj x\+wj*\fk*` through the popover's load and save, in the
  editing views and unformatted.

Tests adapted from Jolie Rabideau's #36.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(shared-react): keep a newly marked span apart from an alike span beside it

In the views that hide marker glyphs, applying a character marker to the
text right after a span of the same marker joined the new span into the
old one: the marker menu or `insertMarker('nd')` on "God" after
`\nd Lord\nd*` saved `\nd LordGod\nd*`. Paratext 9 and Standard view make
a second span.

`getUsjMarkerAction` records every span a marker apply creates
(`$recordNewlyMarkedSpan`), and `$charNodeTransform` keeps a recorded span
apart from an alike span the document already held. The pieces one apply
creates still combine with each other; Enter-then-Backspace rejoining,
the op applier's nested fragments, a paste extending a span, and
`extendCharacterMarker` record nothing and keep combining.

A selection that reaches into part of an alike span and past it now
extends that span instead of nesting a copy inside it. This deliberately
departs from Paratext 9, which mangles the line in that case (TJ,
2026-10-27). Standard view kee…
@lyonsil
lyonsil merged commit 71358e5 into release-prep Nov 4, 2026
3 checks passed
@lyonsil
lyonsil deleted the release-prep-29 branch November 4, 2026 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@tjcouch-sil @lyonsil