refactor(docpdf): hand the HTML-input PDF engines the HTML backend's markup

[devin-ai-integration]

What and why

The PDF backend's HTML-input engines (weasyprint, prince) are handed the HTML document backend's markup — generated from the evaluated document — with a print stylesheet layered over the backend's default sheet, instead of HTML that docpdf wrote itself from a re-parse of the rendered Markdown. docpdf's block parser, page writer and inline translator (markdown.go, html.go, inline.go) are deleted; the package keeps locating and running tools, drawing Mermaid diagrams, typesetting with KaTeX and the typed errors.

• docpdf.Render takes the evaluated *docir.Document; the CLI evaluates once (repl.Session.EvaluateDocument) and hands the IR over. docrender gains Diagrams, Formulas and Captions (what a backend may draw or typeset out of process) and the HTMLOptions.DiagramImages/Math hooks — the one seam where a diagram block becomes its rasterized image, so DOT/PlantUML rasterization can slot in beside mermaid.go without touching it. A table-kind diagram stays a table and takes no image; surplus images are a typed ErrorSurplusDiagramImages.
• The print styling is internal/docpdf/print.css, @layer opensysml-print after @layer opensysml, no style attributes, --sysml-* tokens only (the docrender override-contract assertions now run over it too). A reader's unlayered -html-css sheet therefore overrides PDF exactly as it overrides HTML. -html-theme, -html-no-default-css and -html-css now reach -doc-form pdf; pandoc accepts -html-css (as extra --css) and rejects the other two with a typed error since its page is its own.
• pandoc keeps reading Markdown. Measured: pandoc cannot tell a caption from an emphasized paragraph by syntax, and the marker only ever served it — so the <!-- caption --> marker leaves docrender.Markdown entirely (goldens move by that comment line only, see below) and the pandoc converter's generated Lua filter recognizes captions by matching emphasized paragraphs against docrender.Captions in order, and only ahead of a captionable block (table, display math, diagram fence, the rendering comment a table-kind diagram opens with, grouped table's key). DOT/PlantUML sources get an emphasized notice under pandoc as they do under the HTML engines.
• docrender.Markdown writes a grouped table's key through the shared strong-span writer, so a blank key no longer yields **zone: **, which pandoc set as literal asterisks.
• RenderDocument gains form (markdown/html) and RenderDocumentResponse.html, advertised as capability render_document_html; the Python client's render_document(..., form=) follows. PDF is not offered over the service since it needs the CLI's converter toolchain — recorded as a known limitation.

Docs: docs/manual/outputs.md, docs/reference/cli.md (and the regenerated sysml.1), docs/project/html-document-backend.md (work plan step 2 landed, known limitations), docs/project/spec-compliance.md, .agents/skills/testing-doc-pdf/SKILL.md; changelog fragment changes/unreleased/pdf-onto-html-backend.changed.md.

How it was verified

Gates (all clean): go build ./..., go vet ./..., gofmt -l . (empty), go test ./..., make lint, make docs-check, python3 scripts/changelog.py check, make proto-lint proto-breaking, the Python client tests (985 passed, 91 skipped), and
OPENSYSML_REQUIRE_TRAINING_CORPUS=1 OPENSYSML_REQUIRE_PILOT_CORPORA=1 go test -count=1 ./internal/core/model -run 'TestTrainingExamples|TestPilotCorpora'.

Markdown goldens: git diff origin/develop -- internal/core/docrender/testdata removes 31 lines, every one of them <!-- caption -->; nothing else moves. That includes the state-and-event report golden: its state rows (lamp1.lp in on.fast) and event rows (t=0 lamp1.lp: enter: on) render unchanged, and in PDF they go through the HTML backend's page under print.css like every other row — TestRenderStateReportPage asserts the page handed to the engines is docrender.HTML's (span.sysml-state with data-machine/data-state/data-region, span.sysml-event with data-event-kind/data-time) with the print layer, and TestRenderStateReportWithInstalledEngines reads the state paths, event summaries and 1 [s] instants back out of each installed engine's PDF.

Options an engine cannot take (-html-theme/-html-no-default-css with pandoc) are checked before the engine's presence, so the report is unsupported-option even where pandoc is not installed (TestRenderUnsupportedOptionBeforeTools).

PDF path with the pinned toolchain present

./scripts/download-doc-pdf-toolchain.sh (pandoc 3.10.2, WeasyPrint 69.0, mermaid-cli 11.16.0, KaTeX 0.16.47), then go test -count=1 ./internal/docpdf/... -run 'Installed|Integration|ArtworkFilter' -v with OPENSYSML_WEASYPRINT, OPENSYSML_PANDOC, OPENSYSML_MMDC, OPENSYSML_MMDC_PUPPETEER and OPENSYSML_KATEX set:

--- PASS: TestArtworkFilterMarksCaptionsPastTableRenderings (0.32s)
--- PASS: TestArtworkFilterMarksCaptionOfEmptyTableRendering (0.01s)
--- PASS: TestRenderWithInstalledEngines (1.07s)
    --- PASS: TestRenderWithInstalledEngines/weasyprint (0.50s)
    --- PASS: TestRenderWithInstalledEngines/pandoc (0.55s)
    --- SKIP: TestRenderWithInstalledEngines/prince (0.00s)
--- PASS: TestRenderTelescopeWithInstalledEngines (4.19s)
    --- PASS: TestRenderTelescopeWithInstalledEngines/weasyprint (2.03s)
    --- PASS: TestRenderTelescopeWithInstalledEngines/pandoc (2.09s)
    --- SKIP: TestRenderTelescopeWithInstalledEngines/prince (0.00s)
--- PASS: TestRenderInlineRunsWithInstalledEngines (1.30s)
    --- PASS: TestRenderInlineRunsWithInstalledEngines/weasyprint (0.67s)
    --- PASS: TestRenderInlineRunsWithInstalledEngines/pandoc (0.63s)
    --- SKIP: TestRenderInlineRunsWithInstalledEngines/prince (0.00s)
--- PASS: TestRenderFormulasWithInstalledKatex (3.11s)
    --- PASS: TestRenderFormulasWithInstalledKatex/weasyprint (1.41s)
    --- PASS: TestRenderFormulasWithInstalledKatex/pandoc (1.46s)
    --- SKIP: TestRenderFormulasWithInstalledKatex/prince (0.00s)
--- PASS: TestRenderStateReportWithInstalledEngines (1.39s)
    --- PASS: TestRenderStateReportWithInstalledEngines/weasyprint (0.66s)
    --- PASS: TestRenderStateReportWithInstalledEngines/pandoc (0.73s)
    --- SKIP: TestRenderStateReportWithInstalledEngines/prince (0.00s)
--- PASS: TestRenderDiagramsWithInstalledMermaid (2.10s)
PASS
ok  	github.com/Open-MBEE/OpenSysML/internal/docpdf	13.732s

These extract the PDFs' text and assert: emphasis/strong/code runs render as runs (no *, `, **, ](, <a id literal); the standalone anchor and Ref targets are internal destinations and the external Link a /URI; grouped-table keys are strong headings; the caption is set small while an emphasized body paragraph stays body-sized under both engines, and no {.caption}/[* leaks; KaTeX fonts are embedded and the text carries ∝, λ and none of $, \propto, _{, <span; the Mermaid diagram is an embedded image with its source gone. The two ArtworkFilter tests run the generated Lua filter under pandoc (-t html) over a document mixing table-kind diagrams — whose Markdown puts a <!-- table rendering --> comment between caption and table — with a Mermaid diagram, a table whose caption is padded to CommonMark's indented-code width (" Subsystems by name "), a table whose caption is blank, a further captioned table and an emphasized prose paragraph repeating a caption's text, and assert every caption is marked once and the prose not; both fail with 0 captions marked without the comment recognized as caption evidence. A caption is written as *Subsystems by name* with its surrounding blanks dropped: kept inside the marks (* Subsystems by name *) CommonMark reads literal asterisks, kept outside them four spaces or a tab at block start ( *Subsystems by name*) read as an indented code block, and either way the paragraph never reaches the filter and every later caption falls out of step. docrender.Captions lists the same trimmed text; a blank caption writes no paragraph and is not listed. TestMarkdownPaddedCaption (docrender) pins a caption padded with single blanks, one indented four spaces, one led by a tab, a blank one and a later plain one; the pandoc test fails against the previous renderer with " Subsystems by name " listed untrimmed and the paragraph parsed as code.

Prince was not run — it is proprietary and not provisioned; its subtests skip with the typed not-installed error above. Its converter is verified on the input it is handed: TestRenderHTMLIsTheBackendsPage and the fake-tool tests assert the prepared HTML is docrender.HTML's page (article.sysml-document, sysml-* classes, data-* attributes) with document.css, print.css, KaTeX's sheet and the reader's sheets linked in that order, and the print-stylesheet contract test covers print.css.

Before/after renders, re-adjudicated

Binaries built from origin/develop and this branch rendered the worked example (Observatory::MassReport with -pdf-title-page -pdf-toc -pdf-number-sections), the math report (Optics::OpticsReport) and each document of the linked set (the CLI refuses -render-documents -doc-form pdf, so each was rendered on its own) under weasyprint and pandoc; pdftotext/pdfinfo compared:

document	engine	pages	text
linked: system report	pandoc	1 → 1	identical
linked: appendix	pandoc	1 → 1	identical
linked: system report	weasyprint	1 → 1	before printed the cross-document Ref literally ([Subsystem Masses](Observatory-Mass.20Appendix.md#tables-masses)); after sets it as link text over a /URI to the sibling's HTML page name, as the HTML form writes it. Same words otherwise
linked: appendix	weasyprint	1 → 1	same: [the report](Observatory-SystemReport.md) literal before, link after
math report	weasyprint	2 → 2	identical; KaTeX fonts embedded in both
math report	pandoc	1 → 2	same words; rendered with -doc-toc, and the table of contents now takes a page of its own under pandoc (below)
worked example	weasyprint	5 → 4	same words; the backend's tighter table spacing fits every table on page 3, so "Diagrams" opens page 4 and both Mermaid images fit on it, where before the second one spilled onto a fifth page. Captions now sit under their figure (figcaption) rather than above the block, and the lead paragraph wraps differently
worked example	pandoc	4 → 5	same words in the same order; the table of contents now takes a page of its own (nav#TOC { break-after: page }), as it already did under weasyprint, so every later page number shifts by one

None of the moves changes which text a document carries or its order; the cross-document link is the one behavioral change (a literal Markdown link becoming a link), and a PDF's sibling link pointing at a page file no PDF carries is recorded as a known limitation in html-document-backend.md and spec-compliance.md.

Checklist

• make test and make lint pass locally
• Tests added or updated for the change
• Documentation extended where it already covers the surface (see CONTRIBUTING.md)
• Changelog entry added as changes/unreleased/<slug>.<section>.md, not as an edit to CHANGELOG.md
• baselines regenerated and make docs-counts run if a gate count moved (compliance rows need nothing: the census is counted at docs build)
• No internal work-item labels (waves, slices, F4, K5) in the body, docs, or changelog

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring