fix(format): stop /format from rewriting thematic breaks as list items

[harsh-thakkar7]

Summary

/format rewrote every Markdown thematic break in the document into a list item, turning --- into - - and *** into * **, and destroying any YAML frontmatter block in the process.

Root cause

_apply_markdown_formatting_rules normalises tight list markers (-item → - item). The rule had no notion of a marker run, so it fired on the first two characters of any repetition of -, * or + at the start of a line (backend/ai_logic.py:858, before this change):

line = re.sub(r"^(\s*)([-*+])(\S)", r"\1\2 \3", line)

For --- the regex matches - plus the second - and rewrites the pair to - -. *** likewise became * **. Since this pass runs over the whole document, a single /format invocation silently corrupted all of them.

--- is not an edge case here: it is a standard thematic break (<hr>) and it is also the YAML frontmatter delimiter that this repo parses in backend/knowledge_logic.py (extract_note_title, chunk_markdown_document). The word-count normaliser already special-cases it (backend/word_count.py:28), and tests/test_features.py:96 asserts horizontal-rule support — so it is a first-class construct that /format was destroying. The existing code was also inconsistent with itself: ___ survived untouched, because the rule's character class is [-*+], which excludes _.

Before / after

Request: POST /api/ai/chat with {"message": "format", "document_text": "---\ntitle: My Note\n---\n\nIntro paragraph.\n\n---\n\nOutro paragraph.\n"}

proposed_action.content before this change:

- --
title: My Note
- --

Intro paragraph.

- --

Outro paragraph.

After this change:

---
title: My Note
---

Intro paragraph.

---

Outro paragraph.

The frontmatter is gone in the "before" output; the document is no longer parseable as a note and the intended horizontal rule is a bullet list.

The fix

Require the character after the list marker to be neither another marker nor whitespace, so a marker run stays verbatim while a genuine tight list item is still normalised:

line = re.sub(r"^(\s*)([-*+])(?![-*+])([^\s])", r"\1\2 \3", line)

This is a one-line change to one rule. Heading, ordered-list and blank-line handling are untouched.

Tests

Added TestFormattingRulesPreserveThematicBreaks to tests/test_ai_automation_logic.py, covering thematic breaks (---, ***, indented and spaced variants), frontmatter delimiters, the end-to-end /format automation path, and a guard that tight list items (-item, 1.item, #head) are still normalised exactly as before.

5 of the new tests fail without this fix. Verified by restoring the unfixed file from upstream and re-running:

$ git show origin/main:backend/ai_logic.py > backend/ai_logic.py
$ .venv/bin/python -m pytest tests/test_ai_automation_logic.py -q
FAILED ...::test_format_automation_preserves_thematic_breaks
SUBFAILED(src='Intro\n\n---\n\nOutro') ...::test_thematic_break_is_not_split_into_a_list_item
SUBFAILED(src='para\n\n***\n\npara2')   ...::test_thematic_break_is_not_split_into_a_list_item
SUBFAILED(src='  ---\n')                ...::test_thematic_break_is_not_split_into_a_list_item
FAILED ...::test_yaml_frontmatter_delimiters_are_preserved
5 failed, 13 passed, 8 subtests passed

The test_tight_list_items_are_still_normalised guard passes both before and after, confirming the existing normalisation behaviour is preserved rather than merely removed.

Verification

All CI gates run locally on the branch, against origin/main at 9e77fa0:

Gate	Result
.venv/bin/python -m pytest	199 passed, 24 subtests passed
cd frontend && npm test	23 passed, 0 failed
cd frontend && npm run lint	clean, no output
cd frontend && npm run build	compiled successfully
.venv/bin/ruff format --check (changed files)	2 files already formatted
.venv/bin/ruff check (changed files)	All checks passed

Baseline on origin/main is 195 backend tests and 23 frontend tests, so this PR is +4 backend tests, +0 frontend tests (the new tests are subtests within 4 new test methods; the unfixed run above fails 5 of them).

[Eswar0108]

Review: approve

Verified this independently rather than relying on the PR's own test run.

The fix. The change is a single negative lookahead in
backend/ai_logic.py::_apply_markdown_formatting_rules:

- line = re.sub(r"^(\s*)([-*+])(\S)", r"\1\2 \3", line)
+ line = re.sub(r"^(\s*)([-*+])(?![-*+])([^\s])", r"\1\2 \3", line)

Requiring the third character to be neither whitespace nor another marker is the
right guard: it keeps ---, ***, ***, and - - - verbatim while still
normalising genuine tight list items.

Evidence.

• tests/test_ai_automation_logic.py: 37 passed, 15 subtests passed.
• Reverting only ai_logic.py to upstream/main and keeping the new tests:
  5 failed, 35 passed — test_thematic_break_is_not_split_into_a_list_item
  (3 subtests), test_yaml_frontmatter_delimiters_are_preserved, and
  test_format_automation_preserves_thematic_breaks. The tests genuinely pin the
  behaviour rather than passing vacuously.
• I also re-implemented the rule in isolation and probed it directly. Thematic
  breaks (---, ***, * * *, indented ---, YAML frontmatter delimiters) all
  pass through unchanged; -item, *item, +item, -nested, 1.item, #head
  all still normalise correctly. No regression in the original behaviour.
• CI is green (Backend, Frontend).

Worth calling out: the YAML frontmatter case is the strongest part of this
change. Preserving --- delimiters matters beyond rendering — backend/knowledge_logic.py
parses frontmatter, so a /format run that rewrites a note's delimiters would
corrupt the knowledge base. The test covers that path specifically.

Nit (non-blocking): a run of 5+ markers (-----) is also a valid thematic
break, and it is preserved here as a side effect rather than by intent. That is
the right outcome; just noting the comment explains the 3+ case and the code
happens to cover more.

Ready to merge from my side.