You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Control characters in run text are deleted on save, so Segment.BeginOffset and run boundaries shift after reload #415
Characters XML cannot store (U+0000–U+0008, U+000B, U+000C, U+000E–U+001F, U+FFFE, U+FFFF) are accepted by the model, but TsStringSerializer silently deletes them on write (StripInvalidXmlChars). After reload, anything that stores a character index into the text points at the wrong character.
Take a paragraph One a<U+0001>b two. Three four. whose second segment has BeginOffset 13 ("Three"). Save, close and reopen the project. The text is now 23 characters instead of 24, and BeginOffset 13 points at "hree ". The next parse or edit misplaces analyses.
The same happens with a plain string: runs ab<U+0001>cd (en) + EFGH (es) serialize and deserialize with the es run starting at 4 instead of 5.
ktptOffset is not affected: it is a baseline shift in millipoints, not a character index.
Suggested fix: make writes length-preserving by replacing these characters with U+FFFD instead of deleting them, and apply the same replacement at input in TsStringUtils.NormalizeNfd(ITsString). This is the approach #414 takes for lone surrogates. Whether to replace them or reject them at input is a maintainer decision. Failing tests are ready (ControlCharacterOffsetDesyncTests, and a TsStringSerializerTests round-trip) and can come with the fix PR.
Activity