Add Windows OCR for image-only PDF pages (per-page and batch)

[dan1982code]

OCR for image-only PDF pages (per-page and batch)

For a scanned PDF (images only, no text layer), this adds OCR on Windows using the built-in Windows.Media.Ocr engine — both per-page and in bulk. I know it's only Windows, but hopefully this proof of concept still helps. This feature would be huge for many.

What it does

Per-page (Enter):

• Each PDF page that contains an image but no extractable text now shows a placeholder line instead of the old single document-wide notice:
  [Image only. Press enter to OCR.]
• Pressing Enter while the caret is on that placeholder renders the page to a bitmap via pdfium, runs Windows OCR on a background thread (the UI doesn't freeze), and replaces the placeholder in the document with the recognized text, line by line.
• If OCR finds no text on the page, the placeholder stays and the app announces "No text found."

Batch (Tools → Batch OCR…, default shortcut Ctrl+Shift+O):

• A dialog lets you pick a start and end page, pre-filled with the whole document (1 → last page), and OCRs every image-only page in the range in one go.
• Runs on a background thread with spoken progress every 20 pages ("OCR 40 of 120.") and a final "Batch OCR complete. N pages recognized." — no progress-bar UI, matching the app's screen-reader-first design.
• Pages that already have text (or were already OCR'd) are skipped automatically; the shortcut is remappable in Customize Keyboard Shortcuts like any other action.

What's new under the hood

This introduces the document buffer's first mutation API — until now parsed documents were effectively read-only:

• DocumentBuffer::replace_range (paperback-core) replaces a span in display units, rebuilds the per-char index tables, and shifts markers/id_positions that follow the edit.
• DocumentSession::{replace_range, line_bounds_at, line_text_at} — display-unit-correct line helpers (the existing get_line_text treats its argument as a char index, which misaligns after astral characters).
• A Windows OCR module in the desktop app that fills a WinRT SoftwareBitmap from the pdfium RGBA render and runs OcrEngine on a worker thread, marshalling results back with wxdragon::call_after.
• Batch OCR re-resolves each page's marker position as earlier pages grow, so offsets stay correct even when earlier replacements shift later pages.
• Results are applied in the context of the sliding text window: replaced spans on screen trigger a reload; edits before the current window shift it by the length delta.

Windows OCR notes (read before judging quality)

• We use the default engine with no quality tweaks. Windows.Media.Ocr exposes no preprocessing or accuracy settings, and we deliberately make no attempt to tune it — I don't know anything about tuning that API.
• Pages are rendered to the engine at ~210 DPI (within its ~2600px maximum input dimension; larger inputs are silently downscaled).
• Recognition quality ultimately depends on the installed OCR language packs matching the document's language (the engine uses the user's profile languages; there's no language picker yet).
• The newer NPU-accelerated Windows AI Text Recognition API is reportedly faster and more accurate but requires Copilot+-class hardware (40+ TOPS) plus the Windows App SDK, so it's out of scope here; the engine call is isolated in one function, so swapping it later would be contained.

Not included (out of scope for this pass)

• OCR results are not persisted — reopening the PDF shows the placeholders again. This is how I think it should stay since we don't want to get into saving new PDF files and stuff.
• No UI to choose the OCR language.
• Batch OCR has no cancel button (single-page OCR can't run mid-batch; the app announces "OCR in progress"). What's the best way to do this... replace Batch OCR in the Tools menu with Cancel Batch OCR?

Testing

Tested on Windows 11 with NVDA against an image-only PDF: placeholder announced per image page; Enter runs OCR and the recognized text replaces the placeholder in the richedit; line and page navigation (p) read the OCR'd text; Tools → Batch OCR (Ctrl+Shift+O) OCRs a whole range with spoken progress and skips already-text pages. Windows-only feature.

[dan1982code]

Known limitation: position-based bookmarks (and notes) aren't re-anchored after OCR

Bookmarks are stored as document-absolute display offsets in the per-document .paperback config (StoredBookmark { start, end }), not page-relative. OCR is the first feature that mutates the document buffer, and while replace_range keeps the buffer's own markers and id_positions in sync, it has no visibility into the config's bookmark positions.

Consequence: any bookmark (or note) added before OCR drifts by however much text OCR inserts ahead of its position. A bookmark on a page that gets OCR'd lands at the wrong character offset inside that page's new text — still the right page, but the precise line anchor is lost. OCR-first, bookmark-later is unaffected.

Possible directions:

• Page-relative bookmarks (store page number + offset within the page): bookmarks would survive OCR on unrelated pages, but a bookmark on the page being OCR'd still needs re-anchoring, since its placeholder line is replaced entirely.
• Shift config offsets when OCR inserts text: the OCR apply path already knows the replacement delta — it could shift any bookmarks/positions at or after the replaced span the same way it shifts buffer markers. Contained, and fits the existing architecture.
• Content-based re-anchoring (store the line text and re-find it after any mutation): most robust, biggest change.

Open to direction — happy to take a pass at the delta-shift option if it'd be useful.

[trypsynth]

Hey there,
I had plans for this in a slightly different direction, I'll keep your PR open for now while I see how my preferred method pans out, and merge it if it turns out to flop. Thanks for the work!

[dan1982code]

Thank you! I played around with this a lot before PR'ing. What I ended up liking here was just pressing enter to get any page OCR'd at any time. And I added the Batch OCR to just do the entire document in one shot. Would be curious how your proposal works, and I'd be happy to try to merge any of these ideas in at any point if they end up fitting somehow.

[trypsynth]

Plan, briefly.

We're doing both this and Scribe (our hosted conversion service). They answer different questions, so they get separate paths rather than one abstraction:

• Built-in OS OCR (this PR): local, free, instant, one page or a range. Windows.Media.Ocr on Windows, Vision on macOS. One option in the UI, not a provider list. The platform picks the engine.
• Scribe: remote, paid, whole document, returns real structure (headings, tables, math, image descriptions) as EPUB or DAISY, which our existing parsers already read.

No pluggable provider registry. Two features, two entry points.

So this is getting merged. I'm taking the fixes myself and will push to this branch:

1. Rebase. It's 175 commits behind and conflicting; reload_window_around grew a reason argument since you branched.
2. The batch worker calls render_pdf_page off the UI thread, which render_pdf_page's own doc comment says not to do. It also reopens the PDF once per page.
3. replace_range rebuilds every per-char index table on each call, and batch applies N of them back to back on the UI thread. That's O(n^2) on a long scan.
4. Position drift is wider than you flagged: last_position, navigation_history and temporary_bookmark are absolute offsets too. Taking your delta-shift option and covering all of them.
5. Enter compares the caret's line against t(IMAGE_ONLY_PLACEHOLDER). Change UI language without a reparse and it silently stops matching. Moving to a marker.
6. macOS currently gets the placeholder and then "OCR is only available on Windows." Adding the Vision side so the placeholder means something there.
7. A missing OCR language pack is the likeliest failure for non-English users and currently reports as a generic "OCR failed."
8. Applying batch results per page as they land rather than all at the end. That also gives us cancel almost for free.

Persisting results across reopen is next after that, not a blocker for merging.

The per-page image detection change in pdf.rs is a good fix on its own merit, and line_bounds_at/line_text_at fix a real display-unit bug that was there before OCR. Thanks for this, it's the right shape.

[trypsynth]

Pushed the rebase and the fixes to this branch. 787 core tests pass, the workspace suite passes, clippy and fmt are clean.

First, a correction. I said the batch worker calling render_pdf_page off the UI thread was a thread-safety bug. It isn't. Every pdfium call goes through the crate's lib(), which takes a global reentrant mutex, so access is already serialized. The real cost was reopening the file once per page, which is what I changed.

What landed:

Marker instead of translated text. New MarkerType::ImageOnlyPage, set by the parser on each image-only page. Enter and batch both key off the marker now, so changing UI language can't strand a placeholder. New session API: image_only_pages, image_only_page_at, replace_image_only_pages.

Related: t(IMAGE_ONLY_PLACEHOLDER) passed a constant, and the pot is built by xgettext, which only sees literals. That message was never extracted, so it could not have been translated in any language. It is literal t() calls now and the pot picks up all 23 new strings.

One rebuild per batch, not per page. DocumentBuffer::replace_ranges applies a whole batch and rebuilds the per-char index tables once; replace_range is a one-edit wrapper over it. ReplaceOutcome::shift maps any pre-edit position forward, which is what the config shifting and the caret both use.

Position drift, covering everything. ConfigManager::shift_document_positions remaps last_position, navigation_history, temporary_bookmark and every bookmark range, called after each apply. Your delta-shift option, applied to all four rather than bookmarks alone.

Incremental apply and cancel. The worker opens the PDF once and posts results in groups of 20, so text appears while the job runs instead of all at the end. Choosing Batch OCR while a batch is running asks "Batch OCR is running. Stop it?" rather than needing a second menu item.

macOS. New ui/ocr/macos.rs using Vision through raw objc plus a small CoreGraphics extern block, no new crates. Being straight with you: this compiles nowhere I can test. macOS CI proves it builds; nobody has run it on a Mac yet. If you or anyone reading has one, that is the piece that needs eyes.

Smaller ones. A missing OCR language pack gets its own message instead of a generic "OCR failed." Render size scales from the real page dimensions targeting 300 DPI, capped at whatever the engine reports as its maximum, rather than a flat 1800 px. Linux has no OS OCR engine, so it gets a stub, no menu item and placeholder wording that doesn't promise something Enter can't do.

Not done, and not blocking the merge: OCR results still don't survive reopening the file. The .paperback sidecar is the obvious home for that, keyed by page and file fingerprint. Happy to take it, or leave it to you.

Structure ended up as ui/ocr/{windows,macos,unavailable}.rs behind one dispatcher, with the orchestration in ui/document_manager/ocr.rs. Scribe will not be a provider under that seam, since it works on whole documents and returns structure this shape cannot express. It gets its own path.

[dan1982code]

Awesome... looking forward to test this! This app is on fire :)