diff --git a/docs/breadcrumbs.md b/docs/breadcrumbs.md
index 26ef95c..fbd3522 100644
--- a/docs/breadcrumbs.md
+++ b/docs/breadcrumbs.md
@@ -65,3 +65,229 @@
- Export WebP: added to_webp_or_png() + load_screenshot_optimized() helpers; HTML export uses dynamic MIME; Markdown export converts + writes correct extension.
- Export WebP: removed unused load_screenshot_base64; all 142 Rust + 75 frontend tests pass; clippy clean.
- PDF compression research: WKWebView.createPDF() has zero compression controls; WebP not in PDF spec (re-encoded to lossless = bloat); PDFKit macOS 13+ has saveAllImagesAsJPEG + optimizeImagesForScreen write options; Quartz filters work but need raw FFI; best approach = JPEG data URIs for PDF export + optional PDFKit post-processing. Spec at docs/specs/pdf-compression.md.
+
+2026-02-09
+- Apple Intelligence feasibility research: Foundation Models framework = on-device LLM (3B params, 2-bit), optimized for summarization/extraction/classification; not for deep reasoning/world knowledge.
+- API surface (Swift-first): guided generation (@Generable/@Guide), snapshot streaming, tool calling, stateful sessions; availability gating by Apple-Intelligence device + supported region/language; errors include guardrail violation, unsupported language, context/context-window exceeded.
+- Requirements: Apple Intelligence-capable hardware (Mac M1+), sufficient storage for model download, user must enable Apple Intelligence; Foundation Models framework availability appears tied to macOS 26+ (per Apple Newsroom + third-party Rust binding docs).
+- Mapped StepCast Step fields + AX click metadata (currently only logged) to text-only step description generation; noted that storing AX label/role in Step improves quality without Vision/OCR.
+- Drafted decision matrix: keep Tauri and integrate via small native Swift wrapper (plugin/XPC/CLI) vs hybrid host vs full Swift rewrite; recommendation: keep Tauri, add optional AI path.
+- User decision: ship Apple Intelligence descriptions only on macOS 26+ Apple Silicon; all other cases use deterministic fallback.
+- Research: Apple code-along shows availability cases `.deviceNotEligible`, `.appleIntelligenceNotEnabled`, `.modelNotReady` (+ unknown) via `SystemLanguageModel.default.availability`.
+- Research: adapter deployment requires a separate “FoundationModels Framework Adapter Entitlement”; irrelevant for v1 (default system model only).
+- Design: recommend global in-app toggle (default off) + tutorial note; system toggle lives in macOS Settings “Apple Intelligence & Siri”; no extra TCC permission for text-only descriptions.
+- Implemented: Settings -> Apple Intelligence section with global toggle + privacy note; added “Open Apple Intelligence & Siri Settings” button (deep link + fallback) and a new backend command `get_apple_intelligence_eligibility` (OS/arch check for macOS 26+ + Apple Silicon). Added Quick Start welcome banner bullet.
+- Research: scanned existing FoundationModels OSS usage; found bridge patterns (Swift sidecar JSON stdio, C ABI wrapper) + note that CLI tools may be more strictly rate limited than GUI apps.
+- UI: replaced plain checkbox with an Apple-style native switch using WebKit’s `` (fallback: checkbox when unsupported).
+- Fix: ensure the switch is visible/clickable in StepCast (exclude checkbox/switch inputs from text-input CSS; wrap row in a label to make the whole row clickable). Increased panel height to 640 to avoid Settings view scrolling.
+- Implemented Apple Intelligence step descriptions end-to-end:
+ - Added Swift sidecar helper `src-tauri/swift/stepcast_ai_helper.swift` (FoundationModels) compiled in `src-tauri/build.rs` and executed via JSON stdin/stdout.
+ - Added Tauri commands `update_step_description` + `generate_step_descriptions`; generation updates steps via `step-updated` events.
+ - Recorder: auto-generate missing descriptions after Stop when toggle enabled.
+ - Step Editor: toolbar `Enhance Steps`, per-step sparkle regenerate, inline manual description editing.
+ - Exports: prefer enhanced `step.description` via `effective_description` (HTML/Markdown/PDF).
+ - Settings: switch upgraded to custom Apple-style toggle; tray window auto-resizes for Settings (no scroll).
+- AI quality: added richer AX metadata (subrole/role_description/identifier/container_role) and improved Swift prompt with "UI kind" hints + deterministic baseline + quality gate (prevents vague "Click Finder." and non-tab "tab" hallucinations).
+- AI quality: added on-device Vision OCR on step screenshots (ROI around click) when labels are missing / right-clicks, so descriptions can include sidebar item names + filenames.
+- Fix: AI helper extraction now compares embedded bytes vs existing file (not only length) so updates reliably ship to `~/Library/Caches/com.w0nk1.stepcast/bin/stepcast_ai_helper`.
+- AI quality: AX click metadata is now kept even when the hit-test element has no label; label recovery scans AX children (depth-limited) to pull visible text from list rows (Finder sidebar/file list).
+- AI quality: right-click OCR grounding is filename-only (ignore context menu OCR) and Vision OCR switched to `.accurate` + smaller min text height.
+- AI quality: AX label extraction now prefers `AXTitleUIElement` + `AXPlaceholderValue` and avoids reading `AXValue` for text fields to prevent leaking user-entered content.
+- AI quality: Vision OCR click selection now strongly prefers candidates whose bounding box contains the click (and auto-detects bbox coordinate space vs ROI).
+- AI copy: text-field clicks are described as `Click ... field` (no implicit typing) since StepCast doesn’t record keystrokes.
+- UX: Settings window now auto-resizes to full content using `scrollHeight` + `ResizeObserver` (no Settings scroll).
+- UX: Apple Intelligence toggle now propagates between panel/editor via `ai-toggle-changed` Tauri event (storage event remains as fallback).
+- Verify: `npm test` + `cargo test` pass.
+
+2026-02-10
+- AI quality: AX click grounding now strongly prefers pressable elements (`AXPress`/`AXConfirm`) and attempts to recover labels from child `AXStaticText` on pressable groups; prevents “RustDesk in RustDesk” and improves window control (close) detection.
+- AI quality: store clicked element bounds as `step.ax.element_bounds` (percent) and use it to drive OCR ROI (fallback only); improves label accuracy for buttons/text fields without app-specific heuristics.
+- AI quality: ignore GUID/hex-like AX labels; prefer humanized `AXIdentifier` for grounding when available (e.g. emoji/sticker controls).
+- Capture: treat titleless overlay windows as `Popup` unless near menu bar; avoid top-of-screen region capture for in-app popovers; fixes cropped sticker picker screenshots.
+- Window controls: classify minimize/zoom via AX subrole/roleDescription and emit deterministic baseline (“Minimize the … window.”).
+- Verify: `npm test`, `cargo test`.
+- Packaging: Swift AI helper is now built into `src-tauri/bin/stepcast_ai_helper`, included as a Tauri bundle resource, and executed from the app bundle (no cache extraction).
+- Verify: `cargo build`, `cargo clippy -D warnings`, `cargo test`, `npm test`.
+- Verify: `npm test` + `cargo test` pass.
+- AI quality: for list/table/outline clicks, attempt to replace structural/metadata-heavy AX labels with the currently selected row/item label (AXSelectedRows/AXSelectedChildren); improves Finder file selection and chat list targets without per-app logic.
+- AI quality: list interactions can now be classified using container identifiers (table/list/outline/collection) even when the clicked element role is `AXButton`; improves verbs ("Select …") and reduces "sad" button descriptions.
+- AI quality: common structural labels ("editor area", "scroll area", "split view") are treated as generic and no longer used as grounding labels; reduces cryptic outputs in complex apps like Xcode.
+- Cleanup: fixed new clippy warnings in `src-tauri/src/export/pdf.rs` and `src-tauri/src/recorder/window_info/auth.rs`; ran `cargo fmt` and `cargo clippy -D warnings`.
+
+2026-02-11
+- Investigated capture regression with tiny snippets in `/Users/markus/Library/Caches/com.w0nk1.stepcast/sessions/8fd64097-bfaa-43b6-aefc-d691cdd0d2a9`; found menu-item steps captured via `window_id_capture` despite AX menu roles.
+- Fix: updated titleless overlay classification to allow overlay detection even when frontmost and capture window IDs match; keeps display-relative menu detection.
+- Fix: menu-role AX interactions now force region capture in top menu branch; avoids tiny menu-only screenshots.
+- Fix: popup overlays now union with the largest app window for clicked PID when available, then clamp to clicked display; improves context for picker/popover steps.
+- Verify: `cd src-tauri && cargo check -q` and targeted helper test `classify_titleless_overlay_window_dropdown_vs_popup` pass.
+- Follow-up hardening: restored conservative `same window_id => NotOverlay` classification to avoid false popup classification on titleless utility windows.
+- Follow-up hardening: extracted `should_use_menu_region_capture(...)` + tests; menu-role forcing is now explicitly top-region-gated.
+- Follow-up hardening: added `get_window_for_pid_at_click(...)` and use it for popup union context (fallback remains largest-PID window).
+- Follow-up hardening: added `exclude_window_id` to `get_window_for_pid_at_click(...)` and exclude the active popup window when resolving popup context.
+- Follow-up hardening: popup overlay branch now always captures via region (union bounds) instead of optional window-id capture.
+- Follow-up hardening: `get_window_for_pid_at_click(...)` now chooses the largest click-containing window to avoid selecting another small overlay in front.
+- Follow-up hardening: clamped Dock/menu/top-region capture bounds to clicked display; fixed small-display edge cases where fixed region width/height could exceed display and produce off-screen captures.
+- Verify: full suite green (`cargo test` 165 tests, `npm test` 126 tests).
+- 2026-02-11: audited latest 24-step session (`33edc9e4-d393-4952-bebf-3a8f22c12cf4`) with `scripts/session-ai-report.js` + manual screenshot review; confirmed recurring errors from generic/metadata AX labels (`Seitenleiste`, `Listendarstellung`, `2,4 MB`) and chat metadata labels.
+- 2026-02-11: patched Swift grounding heuristics (`stepcast_ai_helper_descriptions.swift`) to reject structural+metadata labels, improve comma-metadata segment extraction, and avoid wrong right-click labels when OCR has no filename target.
+- 2026-02-11: patched Vision OCR scoring (`stepcast_ai_helper_vision.swift`) to penalize metadata values and prefer filename OCR on list-item double-clicks.
+- 2026-02-11: patched Rust AX list heuristics (`src-tauri/src/recorder/ax_helpers.rs`) to treat sidebar/size/time labels as suspicious and reject those as selected-item targets; added regression tests.
+- 2026-02-11: auth placeholder UX copy updated across recorder/export/editor/timeline to `Authenticate with Touch ID or enter your password to continue.`.
+- 2026-02-11: verification passed (`swiftc -typecheck` for AI helper files, `npm test` 126/126, `cargo test` 167/167 + integration tests).
+- 2026-02-11: after replaying AI generation against latest request JSON, found identifier leak (`NS:8`) and fixed it by filtering implementation-style identifiers from Swift grounding.
+- 2026-02-11: improved OCR filename detection for truncated Finder names (`...` in middle) to recover the actual selected file row on double-click.
+- 2026-02-11: improved `ax_copy_selected_label` to prefer only semantic selected labels and continue searching descendants instead of returning first (often metadata) label.
+- 2026-02-11: deep-dive on latest regression session (`5f8c10ea-7bc0-468d-92fa-296fc7c5e11b`) + prior session (`33edc9e4-d393-4952-bebf-3a8f22c12cf4`) via `recording.log` and `session-ai-report.js`; found repeated false `attached_dialog_window` detections from unrelated apps.
+- 2026-02-11: hardened `src-tauri/src/recorder/window_info/topmost.rs` attached-dialog detection with owner validation + area-ratio constraints; added normalization tests for app-name matching.
+- 2026-02-11: hardened `src-tauri/src/recorder/pipeline/mod.rs` capture selection: normalized same-app checks + safe clicked-window reconciliation by click-point window lookup only.
+- 2026-02-11: verification complete: `cargo fmt`, `cargo test -p stepcast recorder::window_info::topmost -- --nocapture`, `cargo test -p stepcast recorder::pipeline -- --nocapture` all green.
+- 2026-02-11: analyzed newest 28-step session (`0a3241eb-3ccc-476d-a402-5f6f4d0b8309`); identified three regressions: menu-bar opener click ignored, open/save panel click captured as host window, and dialog captures too tightly cropped.
+- 2026-02-11: pipeline fix: stop dropping `AXMenuBarItem` clicks (keep ignoring `AXMenuButton` menu-open noise only).
+- 2026-02-11: pipeline fix: add foreign topmost dialog acceptance (overlap/containment-gated) to support helper-process dialogs (e.g., appkit open/save panels) without reintroducing unrelated behind-window captures.
+- 2026-02-11: pipeline fix: `is_sheet_dialog` now captures parent+dialog union; sheet fast-path now unions AX dialog area with frontmost parent window (display-clamped).
+- 2026-02-11: Swift AI fix: classify `AXMenu` as `menu item` so menu actions are phrased as "Choose ... from the menu".
+- 2026-02-11: verification: `cargo fmt`, `cargo test -p stepcast recorder::pipeline -- --nocapture` (40 pass), `swiftc -typecheck ...` (pass with existing non-blocking warning in vision file).
+- 2026-02-11: audited latest session `d60d83a7-b7c6-4648-9b6e-6e7b07b8e209` (`session-ai-report.js` + `recording.log`) and confirmed regressions at steps 16/21/26 (wrong list target + missing overlay context + low-confidence Arc label).
+- 2026-02-11: patched capture policy in `src-tauri/src/recorder/pipeline/mod.rs` + `helpers.rs` to introduce transient region-capture preference for volatile roles (`AXMenuItem`/`AXMenu`/`AXGroup`) and keep menu-region logic separate/tested.
+- 2026-02-11: added Rust regression tests in `src-tauri/src/recorder/pipeline/helpers.rs` for transient capture policy.
+- 2026-02-11: patched Swift OCR grounding in `stepcast_ai_helper_vision.swift` (no filename-forcing on generic list double-click) and `stepcast_ai_helper_descriptions.swift` (timestamp/date noise filtering + strict baseline fallback without trusted labels).
+- 2026-02-11: replay-validated AI generation locally against session request JSON (`ai-trace-1770817901597-request.json`): step-016 improved to `var 9`; step-026 now deterministic generic fallback (no hallucinated label).
+- 2026-02-11: verification passed: `cargo fmt --all`, `cargo test -p stepcast recorder::pipeline -- --nocapture` (42 pass), `swiftc -typecheck ...` (pass; existing non-blocking Vision warning unchanged).
+- 2026-02-11: fixed pipeline ordering bug in `src-tauri/src/recorder/pipeline/mod.rs`: `AXMenuButton` ignore now runs before `session.next_step_id()`/screenshot path allocation, preventing overwritten `step-XXX.png` files when menu-open is ignored.
+- 2026-02-11: re-verified after ordering fix: `cargo fmt --all`, `cargo test -p stepcast recorder::pipeline -- --nocapture` (45 pass), `swiftc -typecheck ...` (pass; existing non-blocking Vision warning unchanged).
+- 2026-02-11: analyzed regression session `82a47171-cc80-48c5-ac2c-736cc817b591`; found menu-bar clicks misclassified as window-controls (`window_control_fast_path`) and follow-up menu item clicks resolving to underlying `AXGroup` labels (`Today`) from Arc.
+- 2026-02-11: patched `src-tauri/src/recorder/pipeline/mod.rs` + `helpers.rs` with short-lived menu-followup state (`last_menu_bar_click_ms`), menu-role guard for window-control inference, forced `window_title=Menu` on menu-region captures, and tighter transient capture policy (`AXGroup` only for popup overlays).
+- 2026-02-11: patched `src-tauri/swift/stepcast_ai_helper_descriptions.swift` to classify `windowTitle=Menu` top clicks as menu items, improving verb selection (`Choose ... from the menu`).
+- 2026-02-11: patched `src-tauri/swift/stepcast_ai_helper_vision.swift` ROI selection to ignore weak `AXGroup` element bounds on day-like labels (`Today/Heute/...`) and use click-centered ROI.
+- 2026-02-11: verification after this patchset: `cargo fmt --all`, `cargo test -p stepcast recorder::pipeline -- --nocapture` (45 pass), `swiftc -typecheck ...` (pass; existing Vision warning unchanged).
+
+2026-02-12
+- Patched `src-tauri/swift/stepcast_ai_helper_descriptions.swift`: fixed checkbox toggle verb semantics (`checked=true` -> Disable), added file-kind label filtering, and improved OCR cleanup for slash artifacts in filenames.
+- Patched `src-tauri/swift/stepcast_ai_helper.swift`: deterministic baseline for menu/menu-bar/checkbox/window-control kinds; model-error fallback now returns baseline text instead of marking step as failed.
+- Patched `src-tauri/swift/stepcast_ai_helper_vision.swift`: improved list-row OCR selection (row-aligned semantic priority, metadata/file-kind penalties, clipped-vs-full variant preference).
+- Patched `src-tauri/src/recorder/ax_helpers.rs`: for list/outline clicks, try selected-row label more aggressively and retry once after a short delay to catch first-click selection updates.
+- Verification: `swiftc -typecheck ...` pass; helper replay on session `c9f4d328-c2c1-48b3-a119-63a97b510511` now has `FAILURES=0`; `cargo test -p stepcast recorder::ax_helpers -- --nocapture` pass (22/22).
+- Analyzed latest session `c39c68aa-de4f-4489-a5a5-4978b139e131` (`recording.log` + replay): confirmed foreign-overlay capture hijacks (`Grammarly Desktop`) and weak list labels (`Filme`, `hreibtisch`, clipped sidebar strings).
+- Patched `src-tauri/src/recorder/pipeline/mod.rs`: foreign topmost dialog capture now requires explicit AX dialog hints; prevents unrelated helper overlays from replacing the real click target.
+- Patched `src-tauri/swift/stepcast_ai_helper_descriptions.swift`: removed `Ohne Titel`/`Untitled` context suffix, filtered picker-category labels (`Emoji/GIF/Stickers/Today`) as non-targets for item clicks, tightened list-item label acceptance.
+- Patched `src-tauri/swift/stepcast_ai_helper_vision.swift`: added anti-sidebar-leak scoring for list OCR and expanded file-kind detection (plural/localized variants) to avoid column-kind hallucinations.
+- Replay check (`ai-trace-1770891651602-request.json`): improved outputs now include `Enable "WLAN" ...`, removed duplicate app suffix, downgraded unsafe guesses to generic (`Select the item ...`) for weak targets.
+- Verification: `swiftc -typecheck src-tauri/swift/stepcast_ai_helper.swift src-tauri/swift/stepcast_ai_helper_descriptions.swift src-tauri/swift/stepcast_ai_helper_vision.swift`; `cargo test -p stepcast recorder::pipeline -- --nocapture`; `cargo test -p stepcast recorder::ax_helpers -- --nocapture` all pass.
+- Added `docs/specs/2026-02-12-preclick-screencapturekit-buffer.md` and implemented macOS pre-click ScreenCaptureKit ring buffer in `src-tauri/src/recorder/pre_click_buffer.rs`.
+- Integrated pre-click fast-path into `src-tauri/src/recorder/pipeline/mod.rs` for volatile interactions (`prefer_transient_region_capture`) with unchanged fallback to existing capture paths.
+- Wired lifecycle in `src-tauri/src/lib.rs`: start buffer on recording start; stop/release on stop and discard; pass buffer handle into `pipeline::process_click(...)`.
+- Added `src-tauri/build.rs` linker rpath for `/usr/lib/swift` so test/dev binaries resolve Swift runtime dylibs without env overrides.
+- Verification: `cd src-tauri && cargo check -q` pass; `cd src-tauri && cargo test pre_click_buffer::tests::pick_frame -- --nocapture` pass; `cd src-tauri && cargo test -q recorder::pipeline -- --nocapture` pass (45 tests).
+- Updated `src-tauri/src/recorder/pipeline/mod.rs` to prefer pre-click full-display capture for standard clicks (`preclick_fullframe_capture`) before branch-specific window/menu logic.
+- Added stale-frame guard (`PRECLICK_MAX_AGE_MS=500`) and explicit debug logging for `ok/stale/unavailable/failed` pre-click fullframe outcomes.
+- Kept existing capture branches as fallback plus unchanged right-click/auth exceptions.
+- Verification: `cargo fmt --manifest-path src-tauri/Cargo.toml`, `cargo check --manifest-path src-tauri/Cargo.toml`, `cargo test --manifest-path src-tauri/Cargo.toml recorder::pipeline::tests::calculate_click_percent_in_middle -- --nocapture` all pass.
+- Patched `src-tauri/src/recorder/pre_click_buffer.rs`: `pick_frame_index(...)` now returns `None` when all buffered frames are newer than the click timestamp (no fallback to latest/future frame).
+- Patched `src-tauri/src/recorder/pipeline/mod.rs`: pre-click fullframe acceptance now requires `age_ms` in `0..=500` (negative/future frames rejected).
+- Verification: `cargo fmt --manifest-path src-tauri/Cargo.toml`, `cargo check --manifest-path src-tauri/Cargo.toml`, `cargo test --manifest-path src-tauri/Cargo.toml pre_click_buffer::tests::pick_frame -- --nocapture`, `cargo test --manifest-path src-tauri/Cargo.toml recorder::pipeline::tests::calculate_click_percent_in_middle -- --nocapture` all pass.
+- Audited session `d83599f5-b6fb-4b77-8ebe-66de91eb3eb6` via `scripts/session-ai-report.js` + `ai-trace-1770903867743-request/response.json`; confirmed capture path fixed (Poll menu present) but AI-label issues remained (`vat`, `Schreptisch`, `mixuino_errors @ öffnen`, icon glyph ``).
+- Patched `src-tauri/swift/stepcast_ai_helper_descriptions.swift`:
+ - keep `label == window_title` for list/sidebar contexts when container signals list semantics,
+ - strip OCR context-menu suffixes (`@ open/öffnen/...`),
+ - broaden context-menu phrase detection (`offnen`/`oeffnen` variants),
+ - canonicalize `Schreptisch` -> `Schreibtisch`,
+ - strip Private Use Area glyphs from labels,
+ - adjust `preferredVerb(...)` to prioritize actual user action (`Right-click`/`Double-click`/`Press`),
+ - refactor quality gate to return structured decision reason.
+- Patched `src-tauri/swift/stepcast_ai_helper.swift`: response now includes per-step debug payload (`kind`, `location`, `groundingLabel`, `groundingOcr`, `baseline`, `candidate`, `qualityGateReason`).
+- Patched `src-tauri/src/apple_intelligence/mod.rs`: Rust `GenerateResultItem` now accepts optional `debug` field.
+- Patched `src-tauri/src/lib.rs`: persist `ai_desc_debug` lines in `recording.log` for each generated result when debug payload is present.
+- Patched `scripts/session-ai-report.js`: render AI debug summary (`gate`, `label`, `ocr`, `baseline`, `candidate`) when available.
+- Verification:
+ - `cargo fmt --manifest-path src-tauri/Cargo.toml`
+ - `cargo check --manifest-path src-tauri/Cargo.toml`
+ - replayed helper on latest request JSON (`src-tauri/bin/stepcast_ai_helper generate < .../ai-trace-1770903867743-request.json`) and validated:
+ - `step-005`: `Select "Pepper"...` (no `vat`)
+ - `step-030`: `Select "Schreibtisch"...` (typo fixed)
+ - `step-031`: `Right-click "mixuino_errors"...` (no menu contamination)
+ - `step-008/009`: icon glyph removed (`Anonymous Voting` / `Multiple Answers`)
+- Follow-up (user preference): removed language-specific typo canonicalization from `src-tauri/swift/stepcast_ai_helper_descriptions.swift` (no correction list).
+- Follow-up (user preference): strengthened conservative gate to prefer baseline for `list item` / `item` / `button` / `text field` / `tab` whenever candidate differs (`qualityGateReason=prefer_baseline_for_kind`).
+- Re-verified replay on `ai-trace-1770903867743-request.json`: baseline-first behavior active; `step-005` remains stable (`Pepper`), `step-031` remains stable (`mixuino_errors`), and menu steps remain deterministic.
+- Round-2 hardening after reviewing newest session `859e5952-0510-4320-9645-5770a797502e`:
+ - Patched `classifyKind(...)` in `src-tauri/swift/stepcast_ai_helper_descriptions.swift` so `window_title=Menu` only forces `menu item` when AX role is menu-like; fixed false menu wording on `step-002`.
+ - Patched `isGenericUiLabel(...)` to treat `Popover Dismiss Region`/dismiss-region labels as generic; fixed technical identifier output on `step-016`.
+ - Patched list-item OCR gating for sidebar-side clicks (`x < 30%`): reject OCR-only non-filename labels; this intentionally prefers generic baseline over wrong specifics for Finder sidebar-like targets.
+ - Verification: `cargo check --manifest-path src-tauri/Cargo.toml` and replay against `ai-trace-1770979987150-request.json`:
+ - `step-002`: now `Click "Growbox – Home Assistant" in Arc.` (no false menu phrase)
+ - `step-016`: now `Click the item in WhatsApp.` (no `Popover Dismiss Region`)
+ - `step-021/022/023`: Poll flow still correct and stable
+ - `step-030/031/032`: now conservative generic list-item wording instead of wrong OCR names
+- Round-3 hardening (same session): added picker-interaction filter (`sticker`/`emoji`/`gif`/`picker` signals in AX metadata) to suppress OCR/identifier grounding for list items in picker grids.
+- Replay verification (`ai-trace-1770979987150-request.json`):
+ - `step-017` changed from `Select "BIRTH"...` to `Select the item in WhatsApp (Window).`
+ - Poll/menu steps unchanged and still deterministic.
+
+2026-02-13
+- Added spec `docs/specs/2026-02-13-step-crop-focus-system.md` describing non-destructive crop architecture and verification plan.
+- Added `crop_region` to step model (TS + Rust), plus session mutation `update_step_crop` and Tauri command wiring (`update_step_crop`) with permissions/capabilities updates.
+- Implemented recorder auto-focus crop heuristic in `src-tauri/src/recorder/pipeline/helpers.rs` and applied it in main pipeline only for display-sized captures.
+- Implemented editor crop UI using `react-image-crop` in `src/components/EditorStepCard.tsx` with Apply/Reset flow and optimistic state sync via `EditorWindow`.
+- Implemented preview marker remap + cropped rendering in `src/components/EditorStepCard.tsx`, `src/components/StepItem.tsx`, and `src/utils/stepCrop.ts`.
+- Implemented export crop application in `src-tauri/src/export/helpers.rs` and wired into HTML/Markdown paths (PDF inherits HTML path).
+- Added/updated tests: `src/utils/stepCrop.test.ts`, `src/components/StepItem.test.tsx`, `src/components/EditorStepCard.test.tsx`, and Rust crop/marker tests in export/pipeline/session modules.
+- Verification run completed: `npm test`, `npm run build`, `cd src-tauri && cargo test --quiet`, `cd src-tauri && cargo fmt`.
+- Follow-up after reviewing latest sessions/screenshots (`859e...`, `eb821...`): poll/menu captures are now present; biggest remaining readability gap was oversized full-screen-like sheet captures.
+- Patched `src-tauri/src/recorder/pipeline/helpers.rs` with `should_apply_focus_crop(...)` (display-coverage heuristic) and added regression test `should_apply_focus_crop_for_near_fullscreen_or_large_area`.
+- Patched `src-tauri/src/recorder/pipeline/mod.rs` to use focus-crop heuristic in:
+ - `sheet_fast_path` step creation
+ - `window_control_fast_path` step creation
+ - main step finalization (replacing exact-display-size gate)
+- Patched `src-tauri/src/recorder/pre_click_buffer.rs`: `TARGET_FPS` from `12` to `16`.
+- Patched `src-tauri/src/recorder/pipeline/mod.rs`: `PRECLICK_MAX_AGE_MS` from `500` to `250`.
+- Verification follow-up completed:
+ - `cd src-tauri && cargo fmt`
+ - `cd src-tauri && cargo test --quiet recorder::pipeline::helpers::tests::should_apply_focus_crop_for_near_fullscreen_or_large_area`
+ - `npm run -s test`
+ - `npm run -s build`
+ - `cd src-tauri && cargo test --quiet`
+2026-02-13
+- Investigated user-reported missing screenshots for latest recordings ("step 1+2, 4-7").
+- Audited sessions `eb82136b-81f3-44d5-846d-11b3086c434c` and `1fb5073d-0b71-41b4-8e47-61f191cedc28`: screenshot files exist on disk, valid PNGs, and recorder logs show captured paths for the affected step IDs.
+- Implemented frontend hardening: new `mergeUpdatedStep` in `src/utils/stepEvents.ts` and wired into `RecorderPanel` + `EditorWindow` so `step-updated` cannot accidentally drop previously-known screenshot paths.
+- Implemented UI image load hardening in `src/components/StepItem.tsx` and `src/components/EditorStepCard.tsx`: up to 2 delayed retries with cache-busting query on image load failure.
+- Added regression tests `src/utils/stepEvents.test.ts` for omitted/null screenshot update handling.
+- Verification run: `npm run -s test`, `npm run -s build`, `cd src-tauri && cargo test --quiet` all pass.
+2026-02-13
+- Investigated latest user repro: session `3c03f1c9-6d5c-4bcc-89cd-e613446eca47` has all `step-001..015.png` files present and valid; `recording.log` confirms `screenshot_path` emitted for every step.
+- Confirmed issue is editor crop rendering stability, not capture pipeline loss.
+- Patched `src/components/EditorStepCard.tsx`: crop rendering now independent of natural-size readiness, with stable frame fallback ratio and modal image onLoad size hydration.
+- Patched `src/editor.css`: wrapper/frame sizing adjusted to prevent collapse in cropped (absolute image) mode.
+- Added regression test in `src/components/EditorStepCard.test.tsx` for cropped frame visibility.
+- Added spec `docs/specs/2026-02-13-editor-crop-visibility-fix.md`.
+2026-02-13
+- Implemented direct crop reposition in editor overview (`src/components/EditorStepCard.tsx`): pointer-drag on cropped frame updates transient crop preview and persists once on pointer release.
+- Added drag-state guards (`moved` flag + latest crop tracking) so simple clicks do not trigger writes.
+- Added cursor/touch-action styling for draggable cropped frames in `src/editor.css`.
+- Added regression test in `src/components/EditorStepCard.test.tsx` verifying drag updates crop position and preserves crop size.
+- Verification: `npm run -s test -- src/components/EditorStepCard.test.tsx`, `npm run -s test`, `npm run -s build` all pass.
+2026-02-13
+- User-reported crop drag bug reproduced conceptually: native image drag ghost (`+` cursor) appeared while trying to reposition crop.
+- Patched `src/components/EditorStepCard.tsx` to suppress native drag, use pointer-driven crop drag only, and batch pointermove preview updates with `requestAnimationFrame`.
+- Added drag-commit epsilon to avoid writing crop updates on micro-jitter/no-op pointer interactions.
+- Patched `src/editor.css` with `-webkit-user-drag: none` + cropped-image pointer-event policy.
+- Patched `src/components/StepItem.tsx` and `src/components/EditorStepCard.tsx` image tags with lazy/async decode attributes for lower initial load cost.
+- Added spec `docs/specs/2026-02-13-editor-crop-drag-performance.md`.
+- Verification: targeted tests (`EditorStepCard`, `StepItem`), full frontend tests, and build all pass.
+2026-02-14
+- Investigated pre-push blocker: `npm test:coverage` failed on 5 files due per-file thresholds.
+- Added/expanded frontend tests:
+ - `src/components/EditorWindow.test.tsx`
+ - `src/components/EditorStepCard.test.tsx`
+ - `src/components/SettingsSheet.test.tsx`
+ - `src/components/StepItem.test.tsx`
+ - `src/utils/stepCrop.test.ts`
+- Added spec: `docs/specs/2026-02-14-pre-push-coverage-gate.md`.
+- Verification completed with hook-equivalent command:
+ - `sh .githooks/pre-push.sh` (passes: clippy + test:coverage + tsc).
+
+2026-02-14
+- Fixed CI Swift compile blocker (`no such module FoundationModels`) in `src-tauri/swift/stepcast_ai_helper.swift` via conditional import + guarded generation path.
+- Added unsupported availability fallback so `checkAvailability` reports unsupported instead of failing to compile on non-AI runners.
+- Added guarded generation fallback to baseline text with reason `model_unavailable_fallback`.
+- Re-ran hook-equivalent verification: `sh .githooks/pre-push.sh` passes locally.
diff --git a/docs/choices.md b/docs/choices.md
index 1fa5d8c..2d639e4 100644
--- a/docs/choices.md
+++ b/docs/choices.md
@@ -60,3 +60,173 @@
- Default: skip structured logging (log/tracing crates); reason: custom debug_log() + conditional eprintln! is sufficient for current scale.
- Default: add cargo-audit + npm audit to CI; reason: catch known vulnerabilities early.
- Default: keep stale worktrees (codex/fast-capture, codex/permission-gate-settings); reason: branches have unmerged work.
+
+2026-02-09 (apple-intel-feasibility)
+- Default: treat Apple Intelligence step auto-description as optional feature gated by model availability (device/region/language/setting); reason: framework only works on Apple Intelligence-capable devices, must not block recording/export.
+- Default: prefer native Swift wrapper (Tauri plugin/XPC/embedded CLI) over new Rust bindings for first integration; reason: Apple API is Swift-first; third-party Rust bindings are very new/low-adoption and add linkage/rpath risk.
+- Default: avoid “CLI-only” bridges if possible; prefer in-process or GUI-host integration; reason: Apple DTS notes command-line tools can be more strictly rate limited than GUI apps.
+- Default: do not ship Apple Intelligence trademark logos in-app unless Apple provides explicit UI glyph/badge licensing; use generic sparkle icon instead; reason: avoid trademark risk / implied endorsement.
+- Default: use WebKit’s built-in OS-native HTML switch (``) for the toggle UI; reason: matches macOS System Settings look and degrades gracefully to checkbox.
+- Default: increase panel height to `640` to avoid Settings view scrolling; reason: Settings now includes Apple Intelligence section + longer copy.
+- Default: v1 uses text-only inputs (Step fields + captured AX label/role metadata) and does NOT depend on screenshot understanding; reason: avoids OCR/Vision complexity, aligns with device-scale model strengths (summarize/extract/classify).
+- Default: generate AI descriptions post-recording / at export time (not in the click pipeline); reason: avoid adding latency/jitter to recording; simpler error handling; user can export without AI if unavailable.
+- Decision: support envelope for Apple Intelligence descriptions = macOS 26+ + Apple Silicon + Apple Intelligence enabled + supported region/language; otherwise deterministic fallback descriptions; reason: Foundation Models availability/hardware gating.
+- Default: no new macOS TCC permissions needed for basic step description generation; reason: Foundation Models uses system on-device model; only tool-calling into Contacts/Calendar/etc would trigger those standard permissions.
+- Default: do not request/depend on Foundation Models adapter entitlement; reason: adapters are out-of-scope for v1; entitlement needed only for deploying custom adapters.
+- Default: StepCast toggle = global setting (default OFF) with live availability status + explanation; reason: consistent exports, avoids surprises, and feature frequently unavailable on older Macs/regions.
+- Default: tutorial includes a short "Optional: Apple Intelligence descriptions" section (requirements, how to enable in macOS + StepCast, privacy note); reason: reduces support burden and sets correct expectations.
+- Default: open Apple Intelligence settings via `x-apple.systempreferences:com.apple.Siri-Settings.extension` with fallback `x-apple.systempreferences:com.apple.preference.siri`; reason: best-effort deep link to the correct System Settings pane.
+- Default: persist toggle in `localStorage.appleIntelligenceDescriptions`; reason: simple, cross-version, no backend migration needed.
+- Decision: implement Foundation Models integration via embedded Swift sidecar helper (build.rs `swiftc` -> include_bytes -> extract to cache -> JSON stdio); reason: FoundationModels is Swift-first; lowest-risk path for v1 without Rust bindings.
+- Change: toggle UI uses a custom Apple-style switch (button + CSS), not WebKit ``; reason: consistent look/feel in Tauri WKWebView + matches requested UX.
+- Decision: keep tray panel default height at 640 but auto-resize to 720 when Settings is open; reason: avoid scrolling in Settings without adding permanent empty space in the recorder UI.
+- Default: AI description constraints = 1 sentence, max 110 chars, greedy sampling; reason: still short but allows location grounding ("from the Dock", "in Finder").
+- Default: AI descriptions use a deterministic baseline + quality gate (verb/kind/app/label checks) to prevent vague outputs and "tab" hallucinations; reason: user trust > model creativity, stable exports.
+- Default: store additional AX grounding metadata (subrole, role_description, identifier, container_role/subrole) on each step; reason: clicked element is often a child (static text) inside a list row; container hints improve specificity.
+- Default: when AX label missing (or ActionType::RightClick), run on-device Vision OCR on the step screenshot (ROI near click) and use the best near-click text as grounding label; reason: reliably captures sidebar items and filenames (e.g. "Downloads", "coreauth.png") without cloud/LLM vision.
+- Default: never generate AI descriptions for auth placeholder steps or ActionType::Note; reason: avoid sensitive/system dialogs; note steps already user-authored.
+- Default: missing-only/all generation never overwrites manual descriptions; per-step sparkle can overwrite; reason: protect user edits by default.
+- Change: keep AX grounding metadata even when the clicked element has no label; attempt to recover label by scanning the AX child tree (depth-limited); reason: Finder/sidebar/file-list rows often store the visible text on child static text elements.
+- Change: make OCR ROI selection prefer semantic AX sidebar hints; geometry fallback only when AX metadata missing; reason: avoid "left side = sidebar" hacks when AX already provides a clean signal.
+- Change: Vision OCR uses `.accurate` + smaller min text height (0.010); reason: improve label extraction for small UI text in screenshots.
+- Change: Right-click OCR grounding uses filename-like strings only (ignore context menu OCR); reason: prevent wrong targets like "Right-click Open".
+- Change: propagate Apple Intelligence toggle across Tauri webviews via `ai-toggle-changed` event (plus storage fallback); reason: storage events are unreliable in some Tauri window setups.
+- Change: Settings panel auto-resizes to content using `documentElement.scrollHeight` + `ResizeObserver`; reason: no scrolling for the last "eighth" when Settings content grows (eligibility copy, etc.).
+- Change: AX label extraction prefers `AXTitleUIElement` + `AXPlaceholderValue` and avoids `AXValue` for text fields; reason: semantically correct labels without leaking user-entered text.
+- Change: text-field steps baseline uses `Click ... field` (not `Enter ...`); reason: StepCast does not capture global typing events (would require Input Monitoring permission), so claiming typing is inaccurate.
+
+2026-02-10 (ax-pressable-scoring)
+- Change: AX element selection prefers pressable elements (`AXPress`/`AXConfirm`) with a strong minimum score (>=220) and tries to pull the visible label from child `AXStaticText` when pressable; reason: many cross-platform apps expose custom buttons as pressable `AXGroup`s; prevents vague “App in App” steps and improves close/button grounding without OCR or per-app rules.
+- Change: store clicked element bounds (percent within captured screenshot) as `step.ax.element_bounds` and use it to focus OCR ROI when needed; reason: if the click lands on a button background (not on the label text), click-distance OCR can pick the wrong nearby label. Element-bounds ROI keeps OCR “clean” and generic while staying on-device.
+- Change: treat AX labels that look like GUID/hex IDs as non-human and ignore them for grounding; prefer `AXIdentifier`-derived labels over OCR; reason: some apps expose implementation IDs as AX labels (unhelpful for tutorials), and `AXIdentifier` often encodes the real control purpose (e.g. emoji button).
+- Change: window controls (close/minimize/zoom) always use deterministic baseline text (no OCR, no model); reason: OCR picks random chrome text (“Ch”), and the model cannot improve accuracy over the AX subrole signal.
+- Change: titleless overlay windows default to window_title `Popup` (not `Menu`) unless they are near the menu bar; reason: avoids misclassifying in-app popovers/pickers as menus and improves step kind classification downstream.
+
+2026-02-10 (ai-helper-packaging)
+- Change: ship the Swift FoundationModels helper as a bundled resource (`src-tauri/bin/stepcast_ai_helper`) and execute it from the app bundle; reason: avoids running a cache-extracted unsigned binary which can break Gatekeeper/notarization expectations in release builds.
+
+2026-02-10 (list-selection-grounding)
+- Default: for clicks inside list/table/outline containers, prefer the *selected row/item* label (via AXSelectedRows/AXSelectedChildren/AXSelectedItems) when the hit-test label looks structural/metadata-heavy; reason: AppKit lists often return container/view-mode labels at the click point ("List view"), and many apps embed timestamps/status metadata in AX labels. Selected-item label is usually the actual user target (file/chat).
+- Change: classify list interactions using `AX container identifier` / `AX identifier` hints (table/list/outline/collection) even when the clicked element role is `AXButton`/`AXGroup`; reason: many apps implement list rows as pressable buttons, and the verb "Select …" is clearer than "Click … button" for tutorials.
+- Change: treat common structural labels like "editor area"/"scroll area"/"split view" as generic (not grounding labels); reason: avoids "sad" descriptions and keeps output novice-friendly when the UI target isn't a named control.
+
+2026-02-11 (capture-overlay-stability)
+- Change: classify titleless overlays by display-relative Y even when `window_id == main_window_id`; reason: prevents false non-overlay classification on multi-display/menu-overlay cases where frontmost window resolves to the overlay itself.
+- Change: force menu-region capture when AX role is menu-related (`AXMenuItem`/`AXMenuBarItem`/`AXMenuButton`/`AXMenu`); reason: avoids tiny window-ID snippets for status/menu-bar interactions and restores consistent top-region capture.
+- Change: for popup overlays, compute context union using the largest app window for the clicked PID when available; reason: prevents clipped popup-only screenshots (e.g. picker panels) without app-specific hacks.
+
+2026-02-11 (capture-overlay-stability-followup)
+- Change: keep titleless `same window_id` as `NotOverlay`; reason: prevents false popup classification for legitimate titleless utility windows. Menu-bar interactions are now handled via explicit AX menu-role + top-region gating.
+- Change: menu-role forced region capture is limited to top 500px of the clicked display; reason: avoids misclassifying mid-screen context menus as menu-bar captures.
+- Change: popup union now prefers the clicked app’s window at the click point (`get_window_for_pid_at_click`) before falling back to largest PID window; reason: safer in multi-window apps than area-only selection.
+- Change: `get_window_for_pid_at_click` now supports excluding the current popup window id; reason: prevents selecting the popup itself as "context window" and restores intended union-with-base-window behavior.
+- Change: popup overlay path now always uses region capture with computed union bounds; reason: deterministic behavior and fewer tiny overlay-only window-ID captures.
+- Change: `get_window_for_pid_at_click` returns the largest matching window containing the click (not first/front-most); reason: avoids selecting another small overlay and improves base-window context in stacked overlay scenarios.
+- Change: clamp Dock/menu/top-region capture rectangles to clicked display bounds and adapt fixed widths/heights when display is smaller than defaults; reason: avoids off-screen/negative region math and prevents accidental clipped/empty captures on small or offset displays.
+- Change: generic label filtering now rejects structural list labels (`sidebar`/`Seitenleiste`/`source list`/`list`) and metadata value labels (sizes/timestamps) as grounding targets; reason: prevents wrong outputs like "Select Seitenleiste" / "Double-click 2,4 MB".
+- Change: metadata-label simplification now scores comma-separated segments and prefers recipient/entity segments (e.g. `Gesendet an ...`) over preview/status tokens; reason: improves list/chat target naming without app-specific rules.
+- Change: right-click grounding now drops generic labels when OCR cannot recover a filename-like target; reason: safer generic baseline is better than wrong specific text like "Right-click List".
+- Change: auth placeholder steps now carry deterministic description `Authenticate with Touch ID or enter your password to continue.`; reason: clearer guidance for end users and consistent export/editor fallback.
+- Change: ignore implementation-style identifiers like `NS:8` / `AX123` as grounding labels; reason: prevents technical AX internals from leaking into user-facing step text.
+- Change: filename detection now accepts truncated filenames with internal ellipsis (`foo...bar.jpeg`) but still rejects trailing-ellipsis menu items; reason: Finder list OCR often truncates long names.
+- Change: selected-row AX extraction now validates candidates via `selected_label_is_probably_ui_target` before returning and keeps only semantic fallbacks; reason: avoids returning size/status cells when a file/chat target label is available in selected row descendants.
+
+2026-02-11 (capture-owner-hardening)
+- Change: `find_attached_dialog_window` now enforces owner matching (clicked PID/main PID/app-name normalization) and rejects system-UI/fremde app windows; reason: prevent foreign dialogs (Finder/1Password/CleanMyMac/etc.) from hijacking capture target selection.
+- Change: attached-dialog candidates now require sensible size ratio vs main window (4%-95%); reason: reject tiny floating overlays/noise while still allowing real sheets/dialogs.
+- Change: app-name comparisons in capture selection use normalized matching (strip non-alnum + lowercase); reason: avoid false mismatches from hidden Unicode/control chars (e.g. `WhatsApp`).
+- Change: clicked-app reconciliation now switches capture window only when a concrete window for clicked PID exists at click coordinates (`get_window_for_pid_at_click`), no fallback to largest PID window; reason: prevent capturing unrelated windows that were never clicked.
+- Change: do not ignore `AXMenuBarItem` clicks anymore (still ignore `AXMenuButton` menu-open noise); reason: menu-bar opener clicks are meaningful user actions and should be recorded/described.
+- Change: accept foreign topmost windows as dialog candidates when they are non-system, smaller than main window, and strongly overlap/are contained in main window; reason: system-hosted sheets/open-save panels can belong to helper processes and otherwise were lost.
+- Change: for `is_sheet_dialog` captures, force region union of parent window + dialog; reason: better user comprehension than dialog-only crops.
+- Change: sheet fast-path now unions AX dialog region with frontmost parent window (overlap-gated) and clamps to display bounds; reason: preserve dialog context while keeping race-condition robustness.
+- Change: classify `AXMenu` role as `menu item` in Swift description heuristics; reason: actions like attachment menus should use "Choose ... from the menu" instead of generic click text.
+- Change: for volatile AX roles (`AXMenuItem`/`AXMenu`/`AXGroup`), prefer region capture over window-id capture in the normal window path; reason: transient overlays/pickers can disappear before slower window-id capture completes.
+- Change: keep menu-bar region behavior explicit (`should_use_menu_region_capture`) and add separate transient-capture policy (`should_prefer_transient_region_capture`); reason: deterministic, app-agnostic capture selection with clearer testable rules.
+- Change: OCR filename preference is now right-click-only (not all double-click list actions); reason: folders/list items without file extensions (e.g. `var 9`) are valid targets and were previously mis-grounded.
+- Change: relative-day labels (`Today/Heute/Yesterday/Gestern`) and timestamp tokens are treated as non-target metadata in AI grounding; reason: avoid wrong labels like chat date separators.
+- Change: when no trustworthy grounding label exists, force deterministic baseline output (no free model rewrite); reason: prevent hallucinated targets under low-confidence conditions.
+- Change: filter `AXMenuButton` menu-open noise before `next_step_id()` allocation in pipeline; reason: prevents step-id reuse and screenshot overwrite when the following click is the actual menu item.
+- Change: keep a short-lived `last_menu_bar_click_ms` signal and treat near-top follow-up clicks as menu interactions; reason: AX hit-testing on transient menu rows can fall back to underlying app roles (`AXGroup`), causing wrong capture branch selection.
+- Change: restrict transient region capture for `AXGroup` to popup overlays only; reason: generic in-window `AXGroup` clicks should not force volatile capture behavior and can pollute screenshot/context alignment.
+- Change: skip traffic-light geometry fallback for menu roles (`AXMenuBarItem`/`AXMenuItem`/`AXMenu`/`AXMenuButton`); reason: prevents false close/minimize/zoom inference on menu-bar interactions.
+- Change: when menu-region capture is chosen, force step `window_title` to `Menu`; reason: downstream description classification should consistently treat those steps as menu actions.
+- Change: ignore `ax.element_bounds` ROI for menu-like/group hits with weak day-like labels (`Today/Heute/...`) and fall back to click-centered OCR ROI; reason: prevents OCR grounding from drifting into unrelated background text.
+
+2026-02-12 (ai-description-hardening)
+- Change: checkbox verbs now use toggle semantics from current AX state (`checked=true` => `Disable`, `checked=false` => `Enable`); reason: previously steps could be described inverted.
+- Change: menu/menu-bar/checkbox/window-control kinds now use deterministic baseline text (no model call), and model errors fall back to baseline; reason: removes red AI failures for otherwise well-grounded steps.
+- Change: OCR list grounding now penalizes file-kind/status/date metadata labels and prefers row-aligned primary labels; reason: improves filename/folder target selection in Finder-like lists.
+- Change: AX list clicks now attempt selected-row label replacement more aggressively (including one short retry for delayed selection updates); reason: reduce OCR dependence and improve first-click row accuracy.
+
+2026-02-12 (ai-description-hardening-followup)
+- Default: hide generic window titles like `Ohne Titel`/`Untitled` from description context; reason: reduces noise and avoids confusing suffixes.
+- Default: for list-item grounding, reject weak labels that look like file-kind/status/date/sidebar location when click is in content area; reason: better generic fallback than wrong specific target (`Filme`, `hreibtisch`).
+- Default: for popup/group clicks, treat picker category labels (`Emoji/GIF/Stickers/Today`) as non-target context unless role is explicit tab/menu; reason: avoids mislabeling sticker-item clicks as category clicks.
+- Default: if list OCR confidence is low or semantically weak, prefer deterministic generic phrasing (`Select the item...`) over speculative names; reason: stability > false precision.
+
+2026-02-12 (preclick-screencapturekit-buffer)
+- Default: use ScreenCaptureKit pre-click ring buffer only for volatile click paths (`prefer_transient_region_capture`); reason: highest benefit where menus/popovers close fastest, minimal pipeline risk elsewhere.
+- Default: keep existing CG/window/region capture as hard fallback when pre-click frame is unavailable or fails; reason: no regression in current capture baseline.
+- Default: ring buffer depth `4` at `12` FPS with BGRA stream; reason: enough temporal history for menu clicks while keeping CPU/memory bounded.
+- Default: maintain a single active ScreenCaptureKit display stream and switch on display change (first click after switch falls back); reason: avoid multi-stream handler ambiguity in current crate integration.
+- Default: add linker rpath `/usr/lib/swift` in `src-tauri/build.rs`; reason: ensure test/dev binaries can resolve Swift runtime dylibs required by ScreenCaptureKit bridge.
+
+2026-02-12 (pixel-first-capture-default)
+- Default: prefer `preclick_fullframe_capture` for all non-right-click, non-auth interactions; reason: pixel-level full-display frame is the most robust cross-app/web source of truth for transient overlays/menus.
+- Default: keep right-click on existing context-menu path; reason: right-click menus appear after click and would be missed by strict pre-click capture.
+- Default: keep auth dialogs on dedicated secure capture/placeholder path; reason: secure UI may not be capturable via regular full-frame semantics.
+- Default: reject stale pre-click frames older than `500ms`; reason: avoid mismatched screenshots when stream/frame timing drifts.
+- Default: reject pre-click frames captured after the click (`age_ms < 0`) and return no pre-click frame if all ring frames are newer than click timestamp; reason: prevent time-inverted frame selection under queue lag.
+
+2026-02-12 (ai-grounding-debug-telemetry)
+- Default: keep list-item labels equal to `window_title` when AX container hints indicate real list/sidebar targets; reason: avoids false "generic label" rejection that pushed OCR drift (`Pepper` -> `vat`).
+- Default: strip context-menu artifacts from OCR labels (`@ open/öffnen/...`) and broaden context-menu phrase detection; reason: prevent contamination like `"mixuino_errors @ öffnen"` becoming the target label.
+- Default: remove Private Use Area glyphs from labels (e.g. ``); reason: icon-font artifacts are not user-facing text and pollute tutorial steps.
+- Default: quality gate now returns explicit decision reasons (`accepted`, `quoted_label_lost`, etc.); reason: deterministic traceability for why baseline/candidate was chosen.
+- Default: include per-step AI debug payload (`kind`, `groundingLabel`, `groundingOcr`, `baseline`, `candidate`, `qualityGateReason`) in generation response; reason: richer post-mortem data for fragile edge cases.
+
+2026-02-12 (baseline-first-no-typo-list)
+- Default: prefer deterministic baseline text for fragile kinds (`list item`, `item`, `button`, `text field`, `tab`) when model rewrite differs; reason: stability over stylistic rewrites.
+- Default: no language-specific typo correction list in grounding pipeline; reason: keep behavior multilingual and avoid locale-bound rules.
+
+2026-02-12 (result-hardening-round-2)
+- Default: do not classify by `window_title=Menu` alone when AX role is not menu-like; reason: prevents false "Choose ... from the menu" wording on normal web/content clicks.
+- Default: treat infrastructure labels like `Popover Dismiss Region` as generic UI labels; reason: avoid technical/internal identifiers in end-user instructions.
+- Default: for sidebar-side list clicks (`x < 30%`), reject OCR-only labels that are not strong filename tokens; reason: wrong specific names are worse than stable generic steps.
+- Default: for picker-like interactions (emoji/gif/sticker/picker signals in AX role/identifier), reject OCR/identifier labels for list items; reason: grid content OCR is volatile and produces non-actionable fragments.
+
+2026-02-13 (step-focus-crop-system)
+- Default: persist non-destructive per-step crop metadata as `crop_region` percentages (`x/y/width/height`) instead of rewriting screenshots; reason: stable across resolutions, editable, and safe for rollback.
+- Default: auto-focus crop only when capture is display-sized (>= ~screen bounds tolerance); reason: avoid over-cropping already tight window/dialog captures.
+- Default: auto-focus center uses AX element bounds when available, else click position; reason: better first-pass focus without app-specific rules.
+- Default: crop exports by transforming image bytes during export (HTML/Markdown/PDF) while keeping source files untouched; reason: output readability + consistent behavior across formats.
+
+2026-02-13 (step-focus-crop-followup)
+- Default: apply auto-focus crop for very large captures based on display coverage (`area_ratio` / width+height ratios), not only exact fullscreen matches; reason: large sheet/dialog union captures were still too zoomed out for tutorial readability.
+- Default: apply auto-focus crop in fast-path step creation (`sheet_fast_path`, `window_control_fast_path`) as well; reason: these early-return branches bypassed the normal crop decision and produced oversized screenshots.
+- Default: tighten pre-click frame freshness window from `500ms` to `250ms`; reason: reduce stale-frame risk for transient overlays while keeping enough tolerance at current capture FPS.
+- Default: increase pre-click buffer stream rate from `12` to `16` FPS; reason: improve probability of capturing transient UI state close to click-time with acceptable overhead.
+2026-02-13 (screenshot-event-stability)
+- Default: on frontend `step-updated` merges, preserve an existing `screenshot_path` when incoming payload omits it or clears it unexpectedly without `capture_status=Failed`; reason: prevents transient/partial update events from blanking already-captured thumbnails.
+- Default: retry local screenshot image loads up to 2 times with short delay + cache-busting query; reason: reduce timing-related blank thumbnails when filesystem write and webview image decode race.
+2026-02-13 (editor-crop-visibility-fix)
+- Default: cropped editor rendering is activated whenever `crop_region` is valid (not gated on loaded natural image size); reason: avoids post-apply "image disappeared" state caused by sizing race.
+- Default: cropped frame uses a deterministic fallback `aspect-ratio` (`16/9`) until natural image dimensions are known; reason: prevents zero-size layout collapse in absolute-position crop mode.
+- Default: editor screenshot wrapper is block-level and width-constrained (`width: min(100%, 760px)`); reason: stable layout for both full and cropped frames.
+2026-02-13 (editor-crop-drag-reposition)
+- Default: existing crops can be repositioned directly in step preview via pointer drag (left mouse), commit on pointer release only; reason: fast UX without command spam during drag.
+- Default: crop drag uses direct manipulation semantics (content follows cursor), implemented by translating crop origin against pointer delta scaled by crop size; reason: intuitive repositioning of visible area.
+- Default: no-op click (down/up without movement) does not persist crop updates; reason: avoid unnecessary writes/events.
+2026-02-13 (editor-crop-drag-runtime-hardening)
+- Default: disable native browser image drag in editor screenshot surfaces (`draggable=false`, prevent default dragstart, `-webkit-user-drag: none`); reason: avoids ghost-image/plus-cursor interference during crop reposition.
+- Default: crop drag preview updates are batched via `requestAnimationFrame`; reason: smoother drag with lower React render pressure under frequent pointermove events.
+- Default: commit crop changes only when movement exceeds a minimal epsilon (`0.02` percent units); reason: ignore jitter/no-op clicks and reduce unnecessary state writes/events.
+- Default: editor and timeline images use `loading="lazy"` + `decoding="async"`; reason: lower initial decode cost for long recordings.
+2026-02-14 (pre-push-coverage-gate)
+- Default: keep strict per-file thresholds (`80/80/80/80`) and fix via targeted tests instead of lowering config; reason: preserves quality gate and prevents regressions on critical editor/crop flows.
+- Default: cover branch gaps through UI-level tests (RTL/Vitest) before considering instrumentation ignores; reason: executable behavior checks are more trustworthy than coverage-only annotations.
+
+2026-02-14 (swift-foundationmodels-ci-compat)
+- Default: compile AI helper with conditional `FoundationModels` import and feature guards (`#if canImport(FoundationModels)`); reason: keep CI/macOS runners without Apple Intelligence SDK buildable.
+- Default: when `FoundationModels` is unavailable, return deterministic baseline output with explicit reason (`model_unavailable_fallback`) instead of throwing/aborting; reason: preserve functional captures and editor flow without hard failure.
diff --git a/docs/specs/2026-02-09-apple-intelligence-descriptions-toggle.md b/docs/specs/2026-02-09-apple-intelligence-descriptions-toggle.md
new file mode 100644
index 0000000..05978cf
--- /dev/null
+++ b/docs/specs/2026-02-09-apple-intelligence-descriptions-toggle.md
@@ -0,0 +1,43 @@
+# Apple Intelligence Descriptions (Toggle + Generation + Editor UX)
+
+Date: 2026-02-09
+
+## Goal
+
+- Provide a global opt-in toggle for Apple Intelligence-powered step descriptions.
+- Generate concise per-step descriptions on-device using the Foundation Models framework.
+- Communicate requirements + privacy clearly (on-device; no step upload).
+
+## Non-Goals
+
+- Screenshot understanding (OCR/Vision / image model prompts).
+- Tool-calling to external data sources (Contacts/Calendar/etc).
+- Multi-language UI for descriptions (v1 uses concise English prompts).
+
+## UX
+
+- Settings: `Apple Intelligence -> Use for step descriptions` (global toggle, default OFF).
+- Toggle UI: custom Apple-style toggle (button + CSS) to match macOS System Settings look reliably inside Tauri/WKWebView.
+- Eligibility hint (runtime via Foundation Models availability):
+ - Device must support Apple Intelligence.
+ - Apple Intelligence must be enabled in macOS System Settings.
+ - Model may be temporarily not ready (download/initialization).
+- Provide a button to open macOS settings (deep link + fallback):
+ - Primary deep link: `x-apple.systempreferences:com.apple.Siri-Settings.extension`
+ - Fallback: `x-apple.systempreferences:com.apple.preference.siri`
+- Recorder: after stopping a recording, if toggle is ON, auto-generate descriptions for steps that are missing one.
+- Step Editor:
+ - Toolbar button `Enhance Steps` (uses missing-only when missing exists, otherwise regenerates non-manual steps).
+ - Per-step sparkle button to regenerate that step.
+ - Clicking the description text allows inline manual editing (stored as manual source).
+- Quick Start (welcome banner): one bullet mentioning the optional toggle.
+
+## Storage
+
+- `localStorage.appleIntelligenceDescriptions` = `"true" | "false"`
+
+## Privacy Copy (must be precise)
+
+- For Apple Intelligence descriptions: runs on-device and works offline.
+- StepCast does not upload recorded steps or screenshots for this feature (no third-party LLMs).
+- (Do not make broader claims about analytics/update checks.)
diff --git a/docs/specs/2026-02-12-ai-grounding-debug-telemetry.md b/docs/specs/2026-02-12-ai-grounding-debug-telemetry.md
new file mode 100644
index 0000000..aaf8ef2
--- /dev/null
+++ b/docs/specs/2026-02-12-ai-grounding-debug-telemetry.md
@@ -0,0 +1,35 @@
+# Spec: AI Grounding + Debug Telemetry Hardening
+
+Date: 2026-02-12
+Status: implemented
+
+## Problem
+Recent sessions show capture correctness but unstable AI labels:
+- wrong target label from OCR drift (`vat` instead of `Pepper`)
+- context-menu contamination in right-click labels (`@ öffnen`)
+- OCR typo in Finder sidebar (`Schreptisch`)
+- icon-font glyph leakage in labels (``)
+- low observability for why baseline/candidate was chosen
+
+## Goals
+1. Keep high-specificity labels when reliable.
+2. Fall back deterministically when uncertain.
+3. Expose per-step decision data for audits.
+
+## Non-Goals
+- app-specific rule engines
+- complex ML confidence calibration
+
+## Changes
+1. Strengthen label normalization/cleanup (context-menu suffixes, icon glyph filtering, typo canonicalization).
+2. Keep `window_title` label for list/sidebar contexts with list container signals.
+3. Make quality gate return decision reason, not only text.
+4. Include per-step debug payload in AI helper response and recorder logs.
+
+## Acceptance
+- replay latest problematic trace yields:
+ - `step-005` no `vat`
+ - `step-030` no `Schreptisch`
+ - `step-031` no `@ öffnen`
+ - `step-008/009` no icon glyph
+- `cargo check` passes.
diff --git a/docs/specs/2026-02-12-preclick-screencapturekit-buffer.md b/docs/specs/2026-02-12-preclick-screencapturekit-buffer.md
new file mode 100644
index 0000000..a0b5e27
--- /dev/null
+++ b/docs/specs/2026-02-12-preclick-screencapturekit-buffer.md
@@ -0,0 +1,40 @@
+# Pre-Click ScreenCaptureKit Buffer
+
+Date: 2026-02-12
+
+## Problem
+
+Volatile UI (browser/Telegram menus, popovers) closes before current capture path finishes.
+Result: screenshot often misses the clicked menu state; AI label can still be correct from AX, but image context is wrong.
+
+## Goal
+
+Use a short ScreenCaptureKit frame ringbuffer and prefer a pre-click frame for volatile interactions (`AXMenu`, `AXMenuItem`, transient popup/group roles).
+
+## Scope
+
+- Add macOS-only `pre_click_buffer` recorder module.
+- Start buffer when recording starts; stop on stop/discard.
+- In pipeline transient capture branch, attempt pre-click frame first.
+- Keep existing CG/window/region capture as fallback.
+
+## Approach
+
+1. ScreenCaptureKit stream for active display, low-latency settings (BGRA, limited queue depth, capped fps).
+2. Maintain small in-memory ringbuffer with recent frames + capture timestamps.
+3. On volatile click:
+ - Select latest frame at or before click timestamp (fallback: latest frame).
+ - Write frame to current `screenshot_path`.
+ - Use display bounds as capture bounds for click percentages.
+4. If unavailable/failed, continue existing capture logic unchanged.
+
+## Non-Goals
+
+- No OCR/AI prompt changes in this patch.
+- No replacement of existing capture pipeline; only an additional robust fast-path.
+
+## Verification
+
+- Unit tests for frame selection policy (before-click preference + fallback).
+- `cargo test` targeted recorder tests.
+- `cargo check` for full crate compile.
diff --git a/docs/specs/2026-02-13-editor-crop-drag-performance.md b/docs/specs/2026-02-13-editor-crop-drag-performance.md
new file mode 100644
index 0000000..6d4f396
--- /dev/null
+++ b/docs/specs/2026-02-13-editor-crop-drag-performance.md
@@ -0,0 +1,28 @@
+# Spec: Editor Crop Drag Performance & UX Hardening
+
+Date: 2026-02-13
+Owner: codex
+Status: Implemented
+
+## Goal
+- Make drag-based crop repositioning feel immediate and stable.
+- Eliminate native image drag artifacts (ghost image + plus cursor).
+- Keep render cost low for larger recordings.
+
+## Problems Observed
+- Browser native image drag could trigger while user drags crop preview.
+- Crop updates on every pointer event can cause unnecessary re-render churn.
+- Full-size screenshots in long lists increase initial rendering and decode cost.
+
+## Decisions
+- Disable native image drag (`draggable=false`, `onDragStart preventDefault`, `-webkit-user-drag: none`).
+- Keep pointer events on crop frame, not image (`pointer-events: none` on cropped image).
+- Batch pointer-move crop preview with `requestAnimationFrame`.
+- Persist crop only on pointer release and only when movement is meaningful.
+- Use `loading="lazy"` and `decoding="async"` for preview and thumbnail images.
+
+## Validation
+- Unit tests for editor crop interactions remain green.
+- Full frontend test suite remains green.
+- Production build passes.
+
diff --git a/docs/specs/2026-02-13-editor-crop-visibility-fix.md b/docs/specs/2026-02-13-editor-crop-visibility-fix.md
new file mode 100644
index 0000000..00f9e39
--- /dev/null
+++ b/docs/specs/2026-02-13-editor-crop-visibility-fix.md
@@ -0,0 +1,40 @@
+# Spec: Editor Crop Apply Keeps Screenshot Visible
+
+Date: 2026-02-13
+Owner: codex
+Status: Implemented
+
+## Problem
+- After applying a crop in the step editor, the screenshot can disappear.
+- User impact: recorded instructions become unusable because the key visual context is gone.
+
+## Root Cause
+- Cropped mode uses absolute image positioning.
+- Wrapper/frame sizing could collapse (0-size layout path) after crop apply.
+- Crop rendering also depended on loaded natural image size; that introduced a fragile state boundary.
+
+## Requirements
+- Applying a crop must never hide the screenshot.
+- Crop view must render immediately, even if natural image size is not yet known.
+- No destructive image rewrite; UI-only crop state stays in `crop_region`.
+
+## Implementation
+- `EditorStepCard`:
+ - Enable cropped rendering whenever a valid non-full crop exists (independent of natural-size availability).
+ - Add stable frame style with fallback `aspect-ratio` (`16/9`) + `width: 100%`.
+ - Keep natural-size update on regular image load and on crop-modal image load.
+- `editor.css`:
+ - Make editor image wrapper block-level with deterministic width.
+ - Ensure crop frame is `position: relative` and width-constrained.
+ - Prevent collapse when cropped image is absolutely positioned.
+- Regression test:
+ - Added test ensuring cropped frame is rendered with a non-empty aspect ratio style when `crop_region` exists.
+
+## Validation
+- Run frontend tests for `EditorStepCard`.
+- Run full test suite + build.
+- Manual replay:
+ - Open editor.
+ - Apply crop on a step with screenshot.
+ - Verify image remains visible and marker position remains coherent.
+
diff --git a/docs/specs/2026-02-13-step-crop-focus-system.md b/docs/specs/2026-02-13-step-crop-focus-system.md
new file mode 100644
index 0000000..ee73591
--- /dev/null
+++ b/docs/specs/2026-02-13-step-crop-focus-system.md
@@ -0,0 +1,48 @@
+# Spec: Step Focus Crop System
+
+Date: 2026-02-13
+Status: Done
+
+## Goal
+Provide reliable, non-destructive focus crops for recorded step screenshots so guides remain readable and actionable, especially when source captures are full-screen.
+
+## Requirements
+1. Non-destructive: always keep original screenshot file untouched.
+2. Persisted crop metadata per step as percentages (`x/y/width/height`) so crops are resolution-independent.
+3. Recorder should set a sensible default focus crop for very large captures (not only exact full-screen) to keep dialogs/overlays readable.
+4. Editor must allow manual crop adjust + reset-to-full.
+5. Click marker must remain accurate after crop in panel/editor/export.
+6. Export (HTML/Markdown/PDF) must apply crop to image output.
+7. Backward compatibility: steps without crop render/export as full image.
+
+## Data Model
+- Add optional `crop_region: BoundsPercent` to step model (frontend + backend).
+- Validation constraints: clamp values to [0..100], enforce min width/height.
+
+## Recorder Default Focus Heuristic
+Apply when capture bounds are large display-like frames (coverage heuristic against clicked display).
+- Base around click location.
+- If AX element bounds exist, expand around element with margin.
+- Clamp to display bounds in percent.
+- Leave `None` for auth placeholders and already-tight captures.
+
+## Editor UX
+- Add "Crop" action on step card.
+- Open modal cropper with live preview and click marker context.
+- Save => persist `crop_region`.
+- Reset => clear `crop_region` (full image).
+
+## Export Rules
+- If `crop_region` exists, crop image bytes during export pipeline before encoding.
+- Marker coordinates remapped to cropped coordinate space for HTML/PDF overlays.
+- Markdown image output uses cropped bitmap.
+
+## Edge Cases
+- Invalid crop metadata -> ignore crop (fallback full image).
+- Marker outside crop -> hide marker (avoid misleading point).
+- Missing screenshot -> no image/crop rendering.
+
+## Verification
+- Rust unit tests for crop normalization + exported marker remap.
+- Frontend tests for marker remap and crop UI command calls.
+- Existing tests must remain green.
diff --git a/docs/specs/2026-02-14-pre-push-coverage-gate.md b/docs/specs/2026-02-14-pre-push-coverage-gate.md
new file mode 100644
index 0000000..ee2626a
--- /dev/null
+++ b/docs/specs/2026-02-14-pre-push-coverage-gate.md
@@ -0,0 +1,29 @@
+# 2026-02-14 pre-push coverage gate hardening
+
+## Context
+Pre-push hook blocks branch push due per-file 80% coverage thresholds on:
+- `src/components/EditorWindow.tsx`
+- `src/components/EditorStepCard.tsx`
+- `src/components/SettingsSheet.tsx`
+- `src/components/StepItem.tsx`
+- `src/utils/stepCrop.ts`
+
+## Goal
+Raise coverage via behavior tests (no threshold relaxation, no ignore hints), keep runtime logic unchanged.
+
+## Scope
+- Add targeted tests for unexecuted branches:
+ - AI toggle sync/fallback and generation branching
+ - crop modal controls + retry branches
+ - settings fallback/openUrl/toggle branches
+ - thumbnail retry and sortable branches
+ - crop helper utility branches
+- Validate with exact pre-push chain:
+ - `cargo clippy -- -D warnings`
+ - `npm run test:coverage`
+ - `npx tsc --noEmit`
+
+## Non-goals
+- No feature/UI behavior change
+- No runtime code refactor
+- No threshold config changes
diff --git a/docs/step-editor-design-spec.md b/docs/step-editor-design-spec.md
new file mode 100644
index 0000000..b9b6f21
--- /dev/null
+++ b/docs/step-editor-design-spec.md
@@ -0,0 +1,412 @@
+# Step Editor Design Spec
+
+## Research Date: 2026-02-09
+
+## Current State
+
+The editor (`editor.css` + `EditorStepCard.tsx` + `EditorWindow.tsx`) already has a solid foundation:
+- CSS custom properties for light/dark theming
+- 12px border-radius cards with `var(--border)` borders
+- SF Pro system font stack
+- 900px max-width centered layout
+- Correct accent color (`#7C5CFC` / `#A78BFA`)
+
+What follows are concrete, high-impact upgrades derived from analyzing CleanShot X, Scribe, Tango, Linear, Notion, and Apple HIG patterns.
+
+---
+
+## 1. Layout Pattern: Timeline + Cards (Hybrid)
+
+**What the best tools do:**
+- Scribe/Tango: Vertical card stack, each card = one step (screenshot + number + description + note)
+- Linear: Single-column list of dense, aligned cards with 8px spacing scale
+- CleanShot X: Floating annotation overlay with numbered step counters
+
+**Recommendation — "Timeline Cards":**
+
+Add a thin vertical timeline connector on the left side of the step list. Each step card gets a small numbered badge that sits on the timeline line. This creates visual flow and progression without the complexity of a full alternating timeline.
+
+```
+ [1]---[ Step Card: header / note / screenshot ]
+ |
+ [2]---[ Step Card: header / note / screenshot ]
+ |
+ [3]---[ Step Card: header / note / screenshot ]
+```
+
+### CSS Implementation
+
+```css
+/* Timeline container */
+.editor-steps {
+ display: flex;
+ flex-direction: column;
+ gap: 0; /* remove gap, use padding instead */
+ position: relative;
+ padding-left: 48px; /* space for timeline + badge */
+}
+
+/* Vertical connector line */
+.editor-steps::before {
+ content: '';
+ position: absolute;
+ left: 19px; /* center of 40px badge area */
+ top: 0;
+ bottom: 0;
+ width: 2px;
+ background: var(--border);
+ border-radius: 1px;
+}
+
+/* Each step card wrapper */
+.editor-step {
+ position: relative;
+ margin-bottom: 20px;
+}
+
+/* Step number badge on the timeline */
+.editor-step-badge {
+ position: absolute;
+ left: -48px;
+ top: 14px; /* align with header center */
+ width: 28px;
+ height: 28px;
+ border-radius: 50%;
+ background: var(--accent);
+ color: #fff;
+ font-size: 12px;
+ font-weight: 700;
+ display: flex;
+ align-items: center;
+ justify-content: center;
+ z-index: 1;
+ box-shadow: 0 0 0 4px var(--bg-primary); /* "cut through" the line */
+}
+```
+
+**Why this works:** Creates visual progression (users can scan step numbers instantly), feels like a professional documentation tool (Scribe/Tango pattern), and the connecting line gives a sense of flow. The `box-shadow` trick on the badge creates a clean "punch through" effect on the timeline line without extra markup.
+
+---
+
+## 2. Screenshot Display
+
+**What the best tools do:**
+- CleanShot X: `border-radius: 10px`, `box-shadow: 0 10px 30px rgba(0,0,0,0.1)`, screenshots sit on a subtle background
+- Tango: Screenshots have a thin border + slight shadow, displayed at full card width
+- Scribe: Screenshots are full-bleed within the card, annotation highlights drawn on top
+- Linear: Not screenshot-heavy, but media always has consistent rounding
+
+**Recommendation — "Elevated Screenshot":**
+
+```css
+.editor-step-image {
+ padding: 16px;
+ background: var(--bg-secondary);
+ border-radius: 0 0 12px 12px; /* match card bottom corners */
+}
+
+.editor-image-wrapper img {
+ display: block;
+ max-width: 100%;
+ height: auto;
+ border-radius: 8px; /* rounded screenshot corners */
+ border: 1px solid var(--border); /* thin frame */
+ box-shadow:
+ 0 2px 8px rgba(0, 0, 0, 0.06),
+ 0 8px 24px rgba(0, 0, 0, 0.04); /* layered soft shadow */
+}
+
+/* Dark mode: deeper shadow */
+@media (prefers-color-scheme: dark) {
+ .editor-image-wrapper img {
+ box-shadow:
+ 0 2px 8px rgba(0, 0, 0, 0.2),
+ 0 8px 24px rgba(0, 0, 0, 0.15);
+ border-color: rgba(255, 255, 255, 0.06);
+ }
+}
+```
+
+**Key details:**
+- 8px border-radius on screenshots (not too rounded, not sharp)
+- Layered box-shadow (two layers = more depth than a single shadow)
+- 1px border gives definition, especially in light mode where shadows are subtle
+- The `bg-secondary` padding area creates a "stage" for the screenshot (CleanShot X pattern)
+
+---
+
+## 3. Step Number / Header Design
+
+**What the best tools do:**
+- Scribe: Numbered circular badges (purple/blue accent) with bold step text beside them
+- Tango: Step numbers in colored pills, description in regular weight below
+- CleanShot X Counter tool: Numbered circles with white text on colored background
+- Linear: No step numbers, but uses pill-shaped status badges with subtle backgrounds
+
+**Recommendation — Two options (pick one):**
+
+### Option A: Accent Badge (Scribe-style) — Recommended
+
+Already described in the timeline section above. The numbered circle sits on the timeline. Inside the header, remove the redundant "Step N" text and instead show just the action description.
+
+```css
+.editor-step-header {
+ display: flex;
+ align-items: center;
+ gap: 12px;
+ padding: 14px 16px;
+ background: var(--bg-primary); /* same as card, not secondary */
+ border-bottom: 1px solid var(--border);
+}
+
+/* Remove old step-number styling; number is now in the badge */
+.editor-step-desc {
+ font-size: 14px; /* bump up slightly */
+ font-weight: 600;
+ color: var(--text-primary);
+ line-height: 1.3;
+}
+```
+
+### Option B: Inline Pill Badge (Linear-style)
+
+If you prefer no timeline, keep the step number inline but style it as a colored pill:
+
+```css
+.editor-step-number {
+ display: inline-flex;
+ align-items: center;
+ justify-content: center;
+ min-width: 24px;
+ height: 22px;
+ padding: 0 8px;
+ border-radius: 6px;
+ background: rgba(124, 92, 252, 0.12); /* accent at 12% opacity */
+ color: var(--accent);
+ font-size: 11px;
+ font-weight: 700;
+ letter-spacing: 0.02em;
+}
+
+/* Dark mode */
+@media (prefers-color-scheme: dark) {
+ .editor-step-number {
+ background: rgba(167, 139, 250, 0.15);
+ }
+}
+```
+
+**Option A is recommended** because it matches the timeline pattern and creates a more distinctive, premium feel. Option B is simpler to implement if you skip the timeline.
+
+---
+
+## 4. Notes / Annotations Area
+
+**What the best tools do:**
+- Scribe: Inline text below screenshot, editable on click, with a subtle dashed border placeholder
+- Tango: Description text sits above the screenshot, editable inline
+- Notion: Inline editing with a subtle placeholder that says "Empty. Click to edit."
+- Linear: Descriptions use a muted secondary color, become editable with a single click
+
+**Recommendation — "Refined Inline Note":**
+
+Your current implementation is already close. Key refinements:
+
+```css
+.editor-step-note-row {
+ padding: 10px 16px 12px;
+ border-bottom: 1px solid var(--border);
+}
+
+.editor-step-note-btn {
+ display: block;
+ width: 100%;
+ text-align: left;
+ padding: 8px 12px;
+ border: 1px dashed transparent; /* invisible border by default */
+ border-radius: 8px;
+ background: transparent;
+ color: var(--text-secondary);
+ font-size: 13px;
+ font-family: inherit;
+ line-height: 1.5;
+ cursor: pointer;
+ transition: all 0.15s ease;
+ -webkit-app-region: no-drag;
+}
+
+/* Placeholder state */
+.editor-step-note-btn:not(.has-note) {
+ font-style: italic;
+ opacity: 0.6;
+}
+
+/* Has a note — show it as normal text */
+.editor-step-note-btn.has-note {
+ color: var(--text-primary);
+ opacity: 1;
+}
+
+/* Hover reveals the edit affordance */
+.editor-step-note-btn:hover {
+ border-color: var(--border);
+ background: var(--bg-secondary);
+}
+
+.editor-step-note-input {
+ display: block;
+ width: 100%;
+ padding: 8px 12px;
+ border: 1px solid var(--accent);
+ border-radius: 8px;
+ background: var(--bg-primary);
+ color: var(--text-primary);
+ font-size: 13px;
+ font-family: inherit;
+ line-height: 1.5;
+ resize: vertical;
+ outline: none;
+ box-shadow: 0 0 0 3px rgba(124, 92, 252, 0.1); /* focus ring */
+}
+```
+
+**What makes this feel polished:**
+- No visible border until hover (Notion pattern — clean by default)
+- Subtle background shift on hover signals "this is editable"
+- Focus ring using box-shadow (not outline) for consistent cross-browser appearance
+- Italic + reduced opacity for empty placeholder state (visual hierarchy)
+
+---
+
+## 5. Card Container Refinements
+
+**What the best tools do:**
+- CleanShot X: `border-radius: 10px`, light shadows
+- Linear: 8px spacing scale, minimal borders, increased contrast via subtle elevation
+- Modern cards (2025): 12-16px border-radius, layered shadows, 1px borders at 10% opacity
+
+**Recommendation:**
+
+```css
+.editor-step {
+ border: 1px solid var(--border);
+ border-radius: 14px; /* slightly larger for premium feel */
+ overflow: hidden;
+ background: var(--bg-primary);
+ box-shadow: 0 1px 3px rgba(0, 0, 0, 0.04); /* minimal resting elevation */
+ transition: box-shadow 0.2s ease, border-color 0.2s ease;
+}
+
+.editor-step:hover {
+ box-shadow:
+ 0 2px 8px rgba(0, 0, 0, 0.06),
+ 0 8px 24px rgba(0, 0, 0, 0.03);
+ border-color: rgba(0, 0, 0, 0.15); /* slightly stronger border */
+}
+
+/* Dark mode hover */
+@media (prefers-color-scheme: dark) {
+ .editor-step:hover {
+ box-shadow:
+ 0 2px 8px rgba(0, 0, 0, 0.15),
+ 0 8px 24px rgba(0, 0, 0, 0.1);
+ border-color: rgba(255, 255, 255, 0.12);
+ }
+}
+```
+
+**Why:** Hover elevation change gives cards a feeling of interactivity and depth. The transition duration (0.2s) is fast enough to feel responsive but smooth enough to look intentional.
+
+---
+
+## 6. What Makes It Feel "Modern and Polished"
+
+Synthesized from all sources:
+
+### Typography
+- **Use Inter Display or SF Pro Display for the page title** (22px, weight 700). You already use SF Pro via `-apple-system` — good.
+- **Body text at 13-14px** with 1.4-1.5 line-height. Your current 13px is correct.
+- **Step descriptions at 14px weight 600** — slightly larger than body to create hierarchy.
+- **Secondary text at reduced opacity** rather than a completely different color (Linear pattern: use 60-75% opacity of primary text).
+
+### Spacing
+- **8px spacing scale** (Linear's approach): 8, 12, 16, 24, 32, 48. You're already mostly aligned.
+- **24px gap between cards** (upgrade from current 20px for slightly more breathing room).
+- **16px internal padding** on all card sections — consistent horizontal rhythm.
+
+### Color
+- **Neutral-first palette** — you already have this with `--text-secondary` and `--bg-secondary`.
+- **Accent color used sparingly** — only for badges, focus states, and primary actions.
+- **Avoid pure black text on white** — your `#1d1d1f` is perfect (Apple's approach).
+- **Dark mode: increase shadow intensity** 2-3x, reduce border opacity. You're doing this.
+
+### Micro-interactions
+- All interactive elements should have `transition: all 0.15s ease` (you already do this).
+- Consider adding a very subtle `transform: translateY(-1px)` on card hover for "lift" effect.
+- Focus states should use `box-shadow` focus rings, not browser outlines (already in place).
+
+### Overall "Premium" Signals
+1. **Consistent border-radius**: Pick 8, 12, or 14px and use it EVERYWHERE. Mixed radii feel cheap.
+2. **Layered shadows** (two `box-shadow` values): Creates more realistic depth than a single shadow.
+3. **Subtle background differentiation**: `--bg-secondary` for screenshot areas, `--bg-primary` for text areas.
+4. **Generous whitespace**: Better to have too much padding than too little.
+5. **No hard borders between sections** where possible — use background color changes instead.
+
+---
+
+## 7. Dark Mode Considerations
+
+Based on Apple HIG and Linear's approach:
+
+| Element | Light | Dark |
+|---------|-------|------|
+| Card background | `#ffffff` | `#1c1c1e` (current) |
+| Screenshot stage | `#f5f5f7` | `#2c2c2e` (current) |
+| Border | `rgba(0,0,0,0.1)` | `rgba(255,255,255,0.1)` (current) |
+| Shadow intensity | 4-6% opacity | 15-20% opacity |
+| Badge text on accent | `#fff` | `#fff` (stays) |
+| Accent color | `#7C5CFC` | `#A78BFA` (lighter — current) |
+
+Your current dark mode is well-considered. The one upgrade:
+
+```css
+/* Dark mode: add a very subtle inner glow to cards for definition */
+@media (prefers-color-scheme: dark) {
+ .editor-step {
+ box-shadow:
+ inset 0 1px 0 rgba(255, 255, 255, 0.04), /* top edge highlight */
+ 0 1px 3px rgba(0, 0, 0, 0.2);
+ }
+}
+```
+
+This `inset` top-edge highlight is a technique Apple uses in macOS system controls and Linear uses in its cards. It gives a subtle "glass edge" effect in dark mode that prevents cards from looking flat.
+
+---
+
+## 8. Priority Implementation Order (Biggest Impact First)
+
+1. **Screenshot shadow + border** (5 min) — Instant premium feel
+2. **Step number pill/badge styling** (10 min) — Visual identity for steps
+3. **Card hover elevation** (5 min) — Interactivity signal
+4. **Note area refinement** (10 min) — Clean placeholder + focus ring
+5. **Timeline connector** (20 min) — Requires small React changes + CSS
+6. **Dark mode inset highlights** (5 min) — Polish
+7. **Typography fine-tuning** (5 min) — Step description weight/size bump
+
+---
+
+## Sources
+
+- [CleanShot X Features](https://cleanshot.com/features)
+- [How We Redesigned Linear UI](https://linear.app/now/how-we-redesigned-the-linear-ui)
+- [10 Card UI Design Examples 2025](https://bricxlabs.com/blogs/card-ui-design-examples)
+- [Linear Design: The SaaS Design Trend](https://blog.logrocket.com/ux-design/linear-design/)
+- [Accessible Linear Design Dark/Light Modes](https://blog.logrocket.com/how-do-you-implement-accessible-linear-design-across-light-and-dark-modes)
+- [Scribe Step-by-Step Guides](https://scribe.com)
+- [Tango Guide Editor](https://www.tango.ai/)
+- [Apple Dark Mode HIG](https://developer.apple.com/design/human-interface-guidelines/dark-mode)
+- [Flowbite Timeline Component](https://flowbite.com/docs/components/timeline/)
+- [Vertical Timelines with Tailwind CSS](https://cruip.com/3-examples-of-brilliant-vertical-timelines-with-tailwind-css/)
+- [20 Modern UI Design Trends 2025](https://medium.com/@baheer224/20-modern-ui-design-trends-for-developers-in-2025-efdefa5d69e0)
+- [Notion Gallery View Guide](https://super.so/blog/notion-gallery-view-a-comprehensive-guide)
diff --git a/editor.html b/editor.html
new file mode 100644
index 0000000..93368a9
--- /dev/null
+++ b/editor.html
@@ -0,0 +1,14 @@
+
+
+
+
+
+
+ Step Editor
+
+
+
+
+
+
+
diff --git a/package-lock.json b/package-lock.json
index 158e897..b9b975c 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -1,21 +1,25 @@
{
"name": "stepcast",
- "version": "0.2.1",
+ "version": "0.2.2",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "stepcast",
- "version": "0.2.1",
+ "version": "0.2.2",
"dependencies": {
"@aptabase/tauri": "^0.4.1",
+ "@dnd-kit/core": "^6.3.1",
+ "@dnd-kit/sortable": "^10.0.0",
+ "@dnd-kit/utilities": "^3.2.2",
"@tauri-apps/api": "^2",
"@tauri-apps/plugin-dialog": "^2.6.0",
"@tauri-apps/plugin-opener": "^2",
"@tauri-apps/plugin-process": "^2.3.1",
"@tauri-apps/plugin-updater": "^2.10.0",
"react": "^19.1.0",
- "react-dom": "^19.1.0"
+ "react-dom": "^19.1.0",
+ "react-image-crop": "^11.0.10"
},
"devDependencies": {
"@tauri-apps/cli": "^2",
@@ -521,6 +525,59 @@
"node": ">=20.19.0"
}
},
+ "node_modules/@dnd-kit/accessibility": {
+ "version": "3.1.1",
+ "resolved": "https://registry.npmjs.org/@dnd-kit/accessibility/-/accessibility-3.1.1.tgz",
+ "integrity": "sha512-2P+YgaXF+gRsIihwwY1gCsQSYnu9Zyj2py8kY5fFvUM1qm2WA2u639R6YNVfU4GWr+ZM5mqEsfHZZLoRONbemw==",
+ "license": "MIT",
+ "dependencies": {
+ "tslib": "^2.0.0"
+ },
+ "peerDependencies": {
+ "react": ">=16.8.0"
+ }
+ },
+ "node_modules/@dnd-kit/core": {
+ "version": "6.3.1",
+ "resolved": "https://registry.npmjs.org/@dnd-kit/core/-/core-6.3.1.tgz",
+ "integrity": "sha512-xkGBRQQab4RLwgXxoqETICr6S5JlogafbhNsidmrkVv2YRs5MLwpjoF2qpiGjQt8S9AoxtIV603s0GIUpY5eYQ==",
+ "license": "MIT",
+ "dependencies": {
+ "@dnd-kit/accessibility": "^3.1.1",
+ "@dnd-kit/utilities": "^3.2.2",
+ "tslib": "^2.0.0"
+ },
+ "peerDependencies": {
+ "react": ">=16.8.0",
+ "react-dom": ">=16.8.0"
+ }
+ },
+ "node_modules/@dnd-kit/sortable": {
+ "version": "10.0.0",
+ "resolved": "https://registry.npmjs.org/@dnd-kit/sortable/-/sortable-10.0.0.tgz",
+ "integrity": "sha512-+xqhmIIzvAYMGfBYYnbKuNicfSsk4RksY2XdmJhT+HAC01nix6fHCztU68jooFiMUB01Ky3F0FyOvhG/BZrWkg==",
+ "license": "MIT",
+ "dependencies": {
+ "@dnd-kit/utilities": "^3.2.2",
+ "tslib": "^2.0.0"
+ },
+ "peerDependencies": {
+ "@dnd-kit/core": "^6.3.0",
+ "react": ">=16.8.0"
+ }
+ },
+ "node_modules/@dnd-kit/utilities": {
+ "version": "3.2.2",
+ "resolved": "https://registry.npmjs.org/@dnd-kit/utilities/-/utilities-3.2.2.tgz",
+ "integrity": "sha512-+MKAJEOfaBe5SmV6t34p80MMKhjvUz0vRrvVJbPT0WElzaOJ/1xs+D+KDv+tD/NE5ujfrChEcshd4fLn0wpiqg==",
+ "license": "MIT",
+ "dependencies": {
+ "tslib": "^2.0.0"
+ },
+ "peerDependencies": {
+ "react": ">=16.8.0"
+ }
+ },
"node_modules/@esbuild/aix-ppc64": {
"version": "0.27.2",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.2.tgz",
@@ -2773,6 +2830,15 @@
"react": "^19.2.4"
}
},
+ "node_modules/react-image-crop": {
+ "version": "11.0.10",
+ "resolved": "https://registry.npmjs.org/react-image-crop/-/react-image-crop-11.0.10.tgz",
+ "integrity": "sha512-+5FfDXUgYLLqBh1Y/uQhIycpHCbXkI50a+nbfkB1C0xXXUTwkisHDo2QCB1SQJyHCqIuia4FeyReqXuMDKWQTQ==",
+ "license": "ISC",
+ "peerDependencies": {
+ "react": ">=16.13.1"
+ }
+ },
"node_modules/react-is": {
"version": "17.0.2",
"resolved": "https://registry.npmjs.org/react-is/-/react-is-17.0.2.tgz",
@@ -3042,6 +3108,12 @@
"node": ">=20"
}
},
+ "node_modules/tslib": {
+ "version": "2.8.1",
+ "resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
+ "integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
+ "license": "0BSD"
+ },
"node_modules/typescript": {
"version": "5.8.3",
"dev": true,
diff --git a/package.json b/package.json
index 730529c..4672073 100644
--- a/package.json
+++ b/package.json
@@ -7,6 +7,7 @@
"build": "tsc && vite build",
"preview": "vite preview",
"tauri": "tauri",
+ "session:ai-report": "node scripts/session-ai-report.js",
"test": "vitest run",
"test:watch": "vitest",
"test:coverage": "vitest run --coverage",
@@ -20,13 +21,17 @@
},
"dependencies": {
"@aptabase/tauri": "^0.4.1",
+ "@dnd-kit/core": "^6.3.1",
+ "@dnd-kit/sortable": "^10.0.0",
+ "@dnd-kit/utilities": "^3.2.2",
"@tauri-apps/api": "^2",
"@tauri-apps/plugin-dialog": "^2.6.0",
"@tauri-apps/plugin-opener": "^2",
"@tauri-apps/plugin-process": "^2.3.1",
"@tauri-apps/plugin-updater": "^2.10.0",
"react": "^19.1.0",
- "react-dom": "^19.1.0"
+ "react-dom": "^19.1.0",
+ "react-image-crop": "^11.0.10"
},
"devDependencies": {
"@tauri-apps/cli": "^2",
diff --git a/scripts/session-ai-report.js b/scripts/session-ai-report.js
new file mode 100644
index 0000000..b04dc02
--- /dev/null
+++ b/scripts/session-ai-report.js
@@ -0,0 +1,239 @@
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+
+function parseArgs(argv) {
+ const args = { session: null, dir: null, trace: null };
+ for (let i = 2; i < argv.length; i++) {
+ const a = argv[i];
+ if (a === "--session") args.session = argv[++i] ?? null;
+ else if (a === "--dir") args.dir = argv[++i] ?? null;
+ else if (a === "--trace") args.trace = argv[++i] ?? null;
+ else if (a === "-h" || a === "--help") args.help = true;
+ }
+ return args;
+}
+
+function sessionsRoot() {
+ return path.join(os.homedir(), "Library", "Caches", "com.w0nk1.stepcast", "sessions");
+}
+
+function listDirsSortedByMtime(root) {
+ if (!fs.existsSync(root)) return [];
+ const entries = fs.readdirSync(root, { withFileTypes: true });
+ const dirs = entries
+ .filter((e) => e.isDirectory())
+ .map((e) => path.join(root, e.name))
+ .map((p) => ({ p, mtimeMs: fs.statSync(p).mtimeMs }))
+ .sort((a, b) => b.mtimeMs - a.mtimeMs)
+ .map((x) => x.p);
+ return dirs;
+}
+
+function pickSessionDir(args) {
+ if (args.dir) return args.dir;
+ const root = sessionsRoot();
+ if (args.session) return path.join(root, args.session);
+ const dirs = listDirsSortedByMtime(root);
+ return dirs[0] ?? null;
+}
+
+function tryReadJson(filePath) {
+ try {
+ const s = fs.readFileSync(filePath, "utf8");
+ return JSON.parse(s);
+ } catch {
+ return null;
+ }
+}
+
+function listAiTraceFiles(sessionDir) {
+ const files = fs.readdirSync(sessionDir);
+ return files
+ .filter((f) => /^ai-trace-\d+-(request|response)\.json$/.test(f))
+ .map((f) => path.join(sessionDir, f))
+ .map((p) => ({ p, mtimeMs: fs.statSync(p).mtimeMs }))
+ .sort((a, b) => b.mtimeMs - a.mtimeMs)
+ .map((x) => x.p);
+}
+
+function pickTraceTs(sessionDir, args) {
+ if (args.trace) return String(args.trace);
+ const files = listAiTraceFiles(sessionDir);
+ const first = files[0];
+ if (!first) return null;
+ const m = path.basename(first).match(/^ai-trace-(\d+)-/);
+ return m?.[1] ?? null;
+}
+
+function readAiTrace(sessionDir, traceTs) {
+ if (!traceTs) return { request: null, response: null };
+ const req = tryReadJson(path.join(sessionDir, `ai-trace-${traceTs}-request.json`));
+ const res = tryReadJson(path.join(sessionDir, `ai-trace-${traceTs}-response.json`));
+ return { request: req, response: res };
+}
+
+function parseRecordingLog(logPath) {
+ if (!fs.existsSync(logPath)) return { byStepId: new Map(), aiDesc: new Map(), lines: [] };
+ const lines = fs.readFileSync(logPath, "utf8").split("\n").filter(Boolean);
+ const byStepId = new Map();
+ const aiDesc = new Map();
+
+ // Heuristic: events are sequential. We attach the nearest preceding `ax_click` + `click`
+ // to the next `screenshot_path=.../step-XYZ.png` we see.
+ let lastClick = null;
+ let lastAx = null;
+
+ for (const line of lines) {
+ if (line.includes(" click: ")) lastClick = line;
+ if (line.includes(" ax_click: ")) lastAx = line;
+
+ const sp = line.match(/screenshot_path=([^ ]+\/(step-\d+)\.png)/);
+ if (sp) {
+ const stepId = sp[2];
+ const entry = byStepId.get(stepId) ?? { stepId, screenshotPath: sp[1], click: null, ax: null, extra: [] };
+ entry.screenshotPath = sp[1];
+ entry.click = lastClick;
+ entry.ax = lastAx;
+ entry.extra.push(line);
+ byStepId.set(stepId, entry);
+ continue;
+ }
+
+ const ai = line.match(/ai_desc\s+trace=\d+\s+id=([^\s]+)\s+text=(.*)$/);
+ if (ai) {
+ aiDesc.set(ai[1], ai[2]);
+ continue;
+ }
+ const fail = line.match(/ai_desc_failed\s+trace=\d+\s+id=([^\s]+)\s+error=(.*)$/);
+ if (fail) {
+ aiDesc.set(fail[1], `<> ${fail[2]}`);
+ continue;
+ }
+ }
+
+ return { byStepId, aiDesc, lines };
+}
+
+function extractAxSummary(axLine) {
+ if (!axLine) return null;
+ const role = axLine.match(/role=([^ ]+)/)?.[1] ?? "";
+ const label = axLine.match(/label='([^']*)'/)?.[1] ?? "";
+ const parentRole = axLine.match(/parent_role=Some\\(\"([^\"]+)\"\\)/)?.[1] ?? null;
+ const topRole = axLine.match(/top_role=Some\\(\"([^\"]+)\"\\)/)?.[1] ?? null;
+ return { role, label, parentRole, topRole };
+}
+
+function renderReport({ sessionDir, traceTs, aiTrace, log }) {
+ const steps = [];
+
+ // Prefer request.steps order when available (this is the exact set sent to the model).
+ if (aiTrace?.request?.steps && Array.isArray(aiTrace.request.steps)) {
+ for (const s of aiTrace.request.steps) {
+ const stepId = s?.id;
+ if (!stepId) continue;
+ steps.push(stepId);
+ }
+ } else {
+ // Fallback: use screenshot files.
+ const files = fs.existsSync(sessionDir) ? fs.readdirSync(sessionDir) : [];
+ for (const f of files.filter((x) => /^step-\d+\.png$/.test(x)).sort()) {
+ steps.push(f.replace(/\.png$/, ""));
+ }
+ }
+
+ const lines = [];
+ lines.push(`# Session AI Report`);
+ lines.push("");
+ lines.push(`- session_dir: ${sessionDir}`);
+ lines.push(`- trace: ${traceTs ?? "(none found)"}`);
+ lines.push("");
+
+ for (const id of steps) {
+ const entry = log.byStepId.get(id) ?? { stepId: id, screenshotPath: null, click: null, ax: null, extra: [] };
+ if (!entry.screenshotPath) {
+ const fallback = path.join(sessionDir, `${id}.png`);
+ if (fs.existsSync(fallback)) entry.screenshotPath = fallback;
+ }
+ const traceResult = aiTrace?.response?.results?.find?.((r) => r?.id === id) ?? null;
+ const aiText =
+ log.aiDesc.get(id) ??
+ traceResult?.text ??
+ aiTrace?.response?.failures?.find?.((f) => f?.id === id)?.error ??
+ "(no ai output found)";
+
+ const ax = extractAxSummary(entry.ax);
+
+ lines.push(`## ${id}`);
+ lines.push("");
+ lines.push(`- screenshot: ${entry.screenshotPath ?? "(missing)"}${entry.screenshotPath && fs.existsSync(entry.screenshotPath) ? "" : " (not found)"}`);
+ lines.push(`- click_log: ${entry.click ?? "(missing)"}`);
+ lines.push(`- ax_log: ${entry.ax ?? "(missing)"}`);
+ if (ax) {
+ lines.push(`- ax_summary: role=${ax.role} label=${JSON.stringify(ax.label)} parent=${ax.parentRole ?? ""} top=${ax.topRole ?? ""}`);
+ }
+ lines.push(`- ai_text: ${aiText}`);
+ if (traceResult?.debug) {
+ const dbg = traceResult.debug;
+ lines.push(`- ai_debug: kind=${dbg.kind ?? ""} gate=${dbg.qualityGateReason ?? ""} label=${JSON.stringify(dbg.groundingLabel ?? "")} ocr=${JSON.stringify(dbg.groundingOcr ?? "")}`);
+ lines.push(`- ai_baseline: ${dbg.baseline ?? ""}`);
+ if (dbg.candidate != null) lines.push(`- ai_candidate: ${dbg.candidate}`);
+ }
+ lines.push("");
+ }
+
+ // Also include any screenshots that exist but were NOT sent to the model
+ // (e.g. auth placeholder steps), so audits see the full timeline.
+ try {
+ const files = fs.readdirSync(sessionDir).filter((x) => /^step-\d+\.png$/.test(x)).sort();
+ const idsInRequest = new Set(steps);
+ for (const f of files) {
+ const id = f.replace(/\.png$/, "");
+ if (idsInRequest.has(id)) continue;
+ lines.push(`## ${id}`);
+ lines.push("");
+ lines.push(`- screenshot: ${path.join(sessionDir, f)}`);
+ lines.push(`- click_log: (not sent to model)`);
+ lines.push(`- ax_log: (not sent to model)`);
+ lines.push(`- ai_text: (not generated; excluded from AI input)`);
+ lines.push("");
+ }
+ } catch {
+ // ignore
+ }
+
+ return lines.join("\n");
+}
+
+function main() {
+ const args = parseArgs(process.argv);
+ if (args.help) {
+ process.stdout.write(
+ [
+ "Usage:",
+ " node scripts/session-ai-report.js [--dir ] [--session ] [--trace ]",
+ "",
+ "Defaults:",
+ " picks newest session under ~/Library/Caches/com.w0nk1.stepcast/sessions",
+ " picks newest ai-trace-* within that session",
+ "",
+ ].join("\n"),
+ );
+ process.exit(0);
+ }
+
+ const sessionDir = pickSessionDir(args);
+ if (!sessionDir || !fs.existsSync(sessionDir)) {
+ process.stderr.write(`No session dir found.\n`);
+ process.exit(2);
+ }
+
+ const traceTs = pickTraceTs(sessionDir, args);
+ const aiTrace = readAiTrace(sessionDir, traceTs);
+ const log = parseRecordingLog(path.join(sessionDir, "recording.log"));
+
+ process.stdout.write(renderReport({ sessionDir, traceTs, aiTrace, log }));
+ process.stdout.write("\n");
+}
+
+main();
diff --git a/src-tauri/.gitignore b/src-tauri/.gitignore
index b21bd68..83d9aab 100644
--- a/src-tauri/.gitignore
+++ b/src-tauri/.gitignore
@@ -5,3 +5,6 @@
# Generated by Tauri
# will have schema files for capabilities auto-completion
/gen/schemas
+
+# Generated by build.rs (Swift helper; bundled as resource)
+/bin/stepcast_ai_helper
diff --git a/src-tauri/Cargo.lock b/src-tauri/Cargo.lock
index 321fda5..6fcd1fb 100644
--- a/src-tauri/Cargo.lock
+++ b/src-tauri/Cargo.lock
@@ -4048,6 +4048,12 @@ version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
+[[package]]
+name = "screencapturekit"
+version = "1.5.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6ccf069cb109cf8e01ebdca0d55dfce45dbbf669e8c56ed5c62150b056d3ec9f"
+
[[package]]
name = "security-framework"
version = "2.11.1"
@@ -4441,6 +4447,7 @@ dependencies = [
"objc2-foundation",
"objc2-pdf-kit",
"objc2-web-kit",
+ "screencapturekit",
"serde",
"serde_json",
"slug",
diff --git a/src-tauri/Cargo.toml b/src-tauri/Cargo.toml
index 403ac2b..8c97a1d 100644
--- a/src-tauri/Cargo.toml
+++ b/src-tauri/Cargo.toml
@@ -52,6 +52,9 @@ tauri-plugin-global-shortcut = "2"
tokio = { version = "1", features = ["rt-multi-thread"] }
libc = "0.2"
+[target.'cfg(target_os = "macos")'.dependencies]
+screencapturekit = "1.5.0"
+
[dev-dependencies]
tempfile = "3"
diff --git a/src-tauri/build.rs b/src-tauri/build.rs
index 02618b2..647d764 100644
--- a/src-tauri/build.rs
+++ b/src-tauri/build.rs
@@ -1,17 +1,128 @@
fn main() {
+ if std::env::var("CARGO_CFG_TARGET_OS").as_deref() == Ok("macos") {
+ // Ensure Swift runtime dylibs required by ScreenCaptureKit bridge can be resolved
+ // when running test binaries outside an app bundle.
+ println!("cargo:rustc-link-arg=-Wl,-rpath,/usr/lib/swift");
+ }
+
+ build_swift_ai_helper();
+
let manifest = tauri_build::AppManifest::new().commands(&[
"greet",
"check_permissions",
- "request_permissions",
+ "get_apple_intelligence_eligibility",
+ "request_screen_recording",
+ "request_accessibility",
"start_recording",
"pause_recording",
"resume_recording",
"stop_recording",
"get_steps",
+ "update_step_note",
+ "update_step_description",
+ "update_step_crop",
"export_guide",
+ "delete_step",
+ "reorder_steps",
+ "open_editor_window",
"discard_recording",
+ "generate_step_descriptions",
+ "get_startup_state",
+ "mark_startup_seen",
+ "dismiss_whats_new",
]);
let attrs = tauri_build::Attributes::new().app_manifest(manifest);
tauri_build::try_build(attrs).expect("failed to run build script");
}
+
+fn build_swift_ai_helper() {
+ if std::env::var("CARGO_CFG_TARGET_OS").as_deref() != Ok("macos") {
+ return;
+ }
+
+ let manifest_dir =
+ std::path::PathBuf::from(std::env::var("CARGO_MANIFEST_DIR").expect("CARGO_MANIFEST_DIR"));
+ let out_dir = std::path::PathBuf::from(std::env::var("OUT_DIR").expect("OUT_DIR"));
+
+ let swift_dir = manifest_dir.join("swift");
+ let mut sources: Vec = Vec::new();
+ let read_dir = std::fs::read_dir(&swift_dir)
+ .unwrap_or_else(|e| panic!("read swift dir {}: {e}", swift_dir.display()));
+ for entry in read_dir.flatten() {
+ let path = entry.path();
+ if path.extension().and_then(|s| s.to_str()) == Some("swift") {
+ sources.push(path);
+ }
+ }
+ sources.sort();
+ if sources.is_empty() {
+ return;
+ }
+ for src in &sources {
+ println!("cargo:rerun-if-changed={}", src.display());
+ }
+
+ let dst = out_dir.join("stepcast_ai_helper");
+ let resource_dst = manifest_dir.join("bin").join("stepcast_ai_helper");
+
+ // Incremental: skip compiling if the output is newer than the source.
+ let latest_src_mtime = sources
+ .iter()
+ .filter_map(|p| std::fs::metadata(p).ok()?.modified().ok())
+ .max();
+ let mut needs_compile = true;
+ if let (Some(src_mtime), Ok(dst_meta)) = (latest_src_mtime, std::fs::metadata(&dst)) {
+ if let Ok(dst_mtime) = dst_meta.modified() {
+ if dst_mtime >= src_mtime {
+ needs_compile = false;
+ }
+ }
+ }
+
+ if needs_compile {
+ let mut cmd = std::process::Command::new("swiftc");
+ cmd.arg("-parse-as-library").arg("-O");
+ for src in &sources {
+ cmd.arg(src);
+ }
+ let status = cmd
+ .arg("-o")
+ .arg(&dst)
+ .status()
+ .expect("failed to run swiftc");
+
+ if !status.success() {
+ panic!("swiftc failed: {status}");
+ }
+ }
+
+ // Also emit the helper as a bundle resource for notarization/codesigning correctness.
+ // We execute this helper directly from the app bundle in release builds.
+ let resource_dir = resource_dst.parent().expect("resource dst parent");
+ std::fs::create_dir_all(resource_dir)
+ .unwrap_or_else(|e| panic!("create resource dir {}: {e}", resource_dir.display()));
+
+ let should_copy = match (std::fs::read(&dst), std::fs::read(&resource_dst)) {
+ (Ok(a), Ok(b)) => a != b,
+ (Ok(_), Err(_)) => true,
+ (Err(e), _) => panic!("read swift helper output {}: {e}", dst.display()),
+ };
+
+ if should_copy {
+ std::fs::copy(&dst, &resource_dst).unwrap_or_else(|e| {
+ panic!(
+ "copy swift helper {} -> {}: {e}",
+ dst.display(),
+ resource_dst.display()
+ )
+ });
+ }
+
+ #[cfg(target_os = "macos")]
+ {
+ use std::os::unix::fs::PermissionsExt;
+ let perm = std::fs::Permissions::from_mode(0o755);
+ let _ = std::fs::set_permissions(&resource_dst, perm);
+ }
+}
diff --git a/src-tauri/capabilities/default.json b/src-tauri/capabilities/default.json
index 099191c..22d5947 100644
--- a/src-tauri/capabilities/default.json
+++ b/src-tauri/capabilities/default.json
@@ -7,6 +7,7 @@
"core:default",
"macos-permissions:default",
"allow-check-permissions",
+ "allow-get-apple-intelligence-eligibility",
"allow-request-screen-recording",
"allow-request-accessibility",
"allow-start-recording",
@@ -14,8 +15,15 @@
"allow-resume-recording",
"allow-stop-recording",
"allow-get-steps",
+ "allow-update-step-note",
+ "allow-update-step-description",
+ "allow-update-step-crop",
+ "allow-delete-step",
+ "allow-reorder-steps",
+ "allow-open-editor-window",
"allow-export-guide",
"allow-discard-recording",
+ "allow-generate-step-descriptions",
"allow-get-startup-state",
"allow-mark-startup-seen",
"allow-dismiss-whats-new",
diff --git a/src-tauri/capabilities/editor.json b/src-tauri/capabilities/editor.json
new file mode 100644
index 0000000..b25b356
--- /dev/null
+++ b/src-tauri/capabilities/editor.json
@@ -0,0 +1,16 @@
+{
+ "$schema": "../gen/schemas/desktop-schema.json",
+ "identifier": "editor",
+ "description": "Capability for the step editor window",
+ "windows": ["step-editor"],
+ "permissions": [
+ "core:default",
+ "allow-get-steps",
+ "allow-update-step-note",
+ "allow-update-step-description",
+ "allow-update-step-crop",
+ "allow-delete-step",
+ "allow-generate-step-descriptions",
+ "allow-open-editor-window"
+ ]
+}
diff --git a/src-tauri/permissions/autogenerated/delete_step.toml b/src-tauri/permissions/autogenerated/delete_step.toml
new file mode 100644
index 0000000..05a8491
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/delete_step.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-delete-step"
+description = "Enables the delete_step command without any pre-configured scope."
+commands.allow = ["delete_step"]
+
+[[permission]]
+identifier = "deny-delete-step"
+description = "Denies the delete_step command without any pre-configured scope."
+commands.deny = ["delete_step"]
diff --git a/src-tauri/permissions/autogenerated/generate_step_descriptions.toml b/src-tauri/permissions/autogenerated/generate_step_descriptions.toml
new file mode 100644
index 0000000..89c2aa1
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/generate_step_descriptions.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-generate-step-descriptions"
+description = "Enables the generate_step_descriptions command without any pre-configured scope."
+commands.allow = ["generate_step_descriptions"]
+
+[[permission]]
+identifier = "deny-generate-step-descriptions"
+description = "Denies the generate_step_descriptions command without any pre-configured scope."
+commands.deny = ["generate_step_descriptions"]
diff --git a/src-tauri/permissions/autogenerated/get_apple_intelligence_eligibility.toml b/src-tauri/permissions/autogenerated/get_apple_intelligence_eligibility.toml
new file mode 100644
index 0000000..a53be03
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/get_apple_intelligence_eligibility.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-get-apple-intelligence-eligibility"
+description = "Enables the get_apple_intelligence_eligibility command without any pre-configured scope."
+commands.allow = ["get_apple_intelligence_eligibility"]
+
+[[permission]]
+identifier = "deny-get-apple-intelligence-eligibility"
+description = "Denies the get_apple_intelligence_eligibility command without any pre-configured scope."
+commands.deny = ["get_apple_intelligence_eligibility"]
diff --git a/src-tauri/permissions/autogenerated/open_editor_window.toml b/src-tauri/permissions/autogenerated/open_editor_window.toml
new file mode 100644
index 0000000..52ce670
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/open_editor_window.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-open-editor-window"
+description = "Enables the open_editor_window command without any pre-configured scope."
+commands.allow = ["open_editor_window"]
+
+[[permission]]
+identifier = "deny-open-editor-window"
+description = "Denies the open_editor_window command without any pre-configured scope."
+commands.deny = ["open_editor_window"]
diff --git a/src-tauri/permissions/autogenerated/reorder_steps.toml b/src-tauri/permissions/autogenerated/reorder_steps.toml
new file mode 100644
index 0000000..855cf36
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/reorder_steps.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-reorder-steps"
+description = "Enables the reorder_steps command without any pre-configured scope."
+commands.allow = ["reorder_steps"]
+
+[[permission]]
+identifier = "deny-reorder-steps"
+description = "Denies the reorder_steps command without any pre-configured scope."
+commands.deny = ["reorder_steps"]
diff --git a/src-tauri/permissions/autogenerated/update_step_crop.toml b/src-tauri/permissions/autogenerated/update_step_crop.toml
new file mode 100644
index 0000000..aac363e
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/update_step_crop.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-update-step-crop"
+description = "Enables the update_step_crop command without any pre-configured scope."
+commands.allow = ["update_step_crop"]
+
+[[permission]]
+identifier = "deny-update-step-crop"
+description = "Denies the update_step_crop command without any pre-configured scope."
+commands.deny = ["update_step_crop"]
diff --git a/src-tauri/permissions/autogenerated/update_step_description.toml b/src-tauri/permissions/autogenerated/update_step_description.toml
new file mode 100644
index 0000000..b761dbc
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/update_step_description.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-update-step-description"
+description = "Enables the update_step_description command without any pre-configured scope."
+commands.allow = ["update_step_description"]
+
+[[permission]]
+identifier = "deny-update-step-description"
+description = "Denies the update_step_description command without any pre-configured scope."
+commands.deny = ["update_step_description"]
diff --git a/src-tauri/permissions/autogenerated/update_step_note.toml b/src-tauri/permissions/autogenerated/update_step_note.toml
new file mode 100644
index 0000000..e3c424e
--- /dev/null
+++ b/src-tauri/permissions/autogenerated/update_step_note.toml
@@ -0,0 +1,11 @@
+# Automatically generated - DO NOT EDIT!
+
+[[permission]]
+identifier = "allow-update-step-note"
+description = "Enables the update_step_note command without any pre-configured scope."
+commands.allow = ["update_step_note"]
+
+[[permission]]
+identifier = "deny-update-step-note"
+description = "Denies the update_step_note command without any pre-configured scope."
+commands.deny = ["update_step_note"]
diff --git a/src-tauri/src/apple_intelligence/mod.rs b/src-tauri/src/apple_intelligence/mod.rs
new file mode 100644
index 0000000..73b518b
--- /dev/null
+++ b/src-tauri/src/apple_intelligence/mod.rs
@@ -0,0 +1,156 @@
+use crate::recorder::types::Step;
+use serde::{Deserialize, Serialize};
+use std::path::PathBuf;
+use std::sync::OnceLock;
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct AvailabilityResponse {
+ pub available: bool,
+ #[serde(default)]
+ pub reason: Option,
+ #[serde(default)]
+ pub details: Option,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct GenerateRequest {
+ pub steps: Vec,
+ #[serde(default)]
+ pub max_chars: Option,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct GenerateResultItem {
+ pub id: String,
+ pub text: String,
+ #[serde(default)]
+ pub debug: Option,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct GenerateFailureItem {
+ pub id: String,
+ pub error: String,
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct GenerateResponse {
+ pub results: Vec,
+ pub failures: Vec,
+}
+
+#[cfg(target_os = "macos")]
+static AI_HELPER_PATH: OnceLock = OnceLock::new();
+
+#[cfg(target_os = "macos")]
+pub fn init(app: &tauri::AppHandle) -> Result<(), String> {
+ use tauri::path::BaseDirectory;
+ use tauri::Manager;
+
+ let path = app
+ .path()
+ .resolve("bin/stepcast_ai_helper", BaseDirectory::Resource)
+ .map_err(|e| format!("resolve ai helper resource: {e}"))?;
+ if !path.exists() {
+ return Err(format!("missing ai helper resource at {}", path.display()));
+ }
+ AI_HELPER_PATH
+ .set(path)
+ .map_err(|_| "ai helper path already initialized".to_string())?;
+ Ok(())
+}
+
+#[cfg(target_os = "macos")]
+fn helper_path() -> Result<&'static PathBuf, String> {
+ if let Some(p) = AI_HELPER_PATH.get() {
+ return Ok(p);
+ }
+
+ // Best-effort fallback (dev/test): derive from the current executable location.
+ let exe = std::env::current_exe().map_err(|e| format!("current_exe: {e}"))?;
+ // StepCast.app/Contents/MacOS/StepCast -> StepCast.app/Contents/Resources/bin/stepcast_ai_helper
+ let candidate = exe
+ .parent()
+ .and_then(|p| p.parent())
+ .map(|p| p.join("Resources").join("bin").join("stepcast_ai_helper"))
+ .ok_or("could not derive helper path".to_string())?;
+
+ if !candidate.exists() {
+ return Err(format!(
+ "ai helper not initialized and fallback path missing: {}",
+ candidate.display()
+ ));
+ }
+ let _ = AI_HELPER_PATH.set(candidate);
+ Ok(AI_HELPER_PATH.get().expect("just set"))
+}
+
+#[cfg(target_os = "macos")]
+fn run_helper(args: &[&str], stdin: Option<&[u8]>) -> Result, String> {
+ use std::io::Write;
+ use std::process::{Command, Stdio};
+
+ let helper = helper_path()?;
+ let mut cmd = Command::new(helper);
+ cmd.args(args).stdout(Stdio::piped()).stderr(Stdio::piped());
+
+ if stdin.is_some() {
+ cmd.stdin(Stdio::piped());
+ } else {
+ cmd.stdin(Stdio::null());
+ }
+
+ let mut child = cmd.spawn().map_err(|e| format!("spawn ai helper: {e}"))?;
+ if let Some(input) = stdin {
+ if let Some(mut w) = child.stdin.take() {
+ w.write_all(input)
+ .map_err(|e| format!("write helper stdin: {e}"))?;
+ }
+ }
+
+ let out = child
+ .wait_with_output()
+ .map_err(|e| format!("wait ai helper: {e}"))?;
+ if !out.status.success() {
+ let stderr = String::from_utf8_lossy(&out.stderr);
+ return Err(format!(
+ "ai helper failed ({}): {}",
+ out.status,
+ stderr.trim()
+ ));
+ }
+ Ok(out.stdout)
+}
+
+#[cfg(not(target_os = "macos"))]
+fn run_helper(_args: &[&str], _stdin: Option<&[u8]>) -> Result, String> {
+ Err("not supported on this platform".into())
+}
+
+pub fn availability() -> Result {
+ let out = run_helper(&["availability"], None)?;
+ serde_json::from_slice(&out).map_err(|e| format!("parse availability json: {e}"))
+}
+
+pub fn generate_descriptions(
+ steps: Vec,
+ max_chars: usize,
+) -> Result {
+ // Keep the Swift helper API stable: snake_case JSON.
+ let req = GenerateRequest {
+ steps,
+ max_chars: Some(max_chars),
+ };
+ let input = serde_json::to_vec(&req).map_err(|e| format!("encode generate json: {e}"))?;
+ let out = run_helper(&["generate"], Some(&input))?;
+ serde_json::from_slice(&out).map_err(|e| format!("parse generate json: {e}"))
+}
+
+pub fn is_auth_placeholder(step: &Step) -> bool {
+ step.window_title == "Authentication dialog (secure)"
+ || step.app.to_lowercase() == "authentication"
+}
+
+pub fn is_blank_description(desc: Option<&str>) -> bool {
+ desc.unwrap_or("").trim().is_empty()
+}
diff --git a/src-tauri/src/export/helpers.rs b/src-tauri/src/export/helpers.rs
index 49c0454..dac9f43 100644
--- a/src-tauri/src/export/helpers.rs
+++ b/src-tauri/src/export/helpers.rs
@@ -1,4 +1,4 @@
-use crate::recorder::types::{ActionType, Step};
+use crate::recorder::types::{ActionType, BoundsPercent, Step};
use base64::Engine;
use std::fs;
@@ -8,10 +8,104 @@ pub fn is_auth_placeholder(step: &Step) -> bool {
|| step.app.to_lowercase() == "authentication"
}
+fn normalize_crop_region(crop_region: Option<&BoundsPercent>) -> Option {
+ let crop = crop_region?;
+ let values = [
+ crop.x_percent,
+ crop.y_percent,
+ crop.width_percent,
+ crop.height_percent,
+ ];
+ if values.iter().any(|v| !v.is_finite()) {
+ return None;
+ }
+
+ let x = crop.x_percent.clamp(0.0, 100.0);
+ let y = crop.y_percent.clamp(0.0, 100.0);
+ let mut w = crop.width_percent.clamp(0.0, 100.0);
+ let mut h = crop.height_percent.clamp(0.0, 100.0);
+ if x + w > 100.0 {
+ w = (100.0 - x).max(0.0);
+ }
+ if y + h > 100.0 {
+ h = (100.0 - y).max(0.0);
+ }
+ if w < 2.0 || h < 2.0 {
+ return None;
+ }
+
+ Some(BoundsPercent {
+ x_percent: x,
+ y_percent: y,
+ width_percent: w,
+ height_percent: h,
+ })
+}
+
+fn crop_rect_px(
+ img_w: u32,
+ img_h: u32,
+ crop_region: Option<&BoundsPercent>,
+) -> Option<(u32, u32, u32, u32)> {
+ let crop = normalize_crop_region(crop_region)?;
+ if crop.x_percent <= 0.0
+ && crop.y_percent <= 0.0
+ && crop.width_percent >= 99.99
+ && crop.height_percent >= 99.99
+ {
+ return None;
+ }
+
+ let w_f = img_w as f64;
+ let h_f = img_h as f64;
+ let x = ((crop.x_percent as f64 / 100.0) * w_f).floor() as u32;
+ let y = ((crop.y_percent as f64 / 100.0) * h_f).floor() as u32;
+ let mut width = ((crop.width_percent as f64 / 100.0) * w_f).round() as u32;
+ let mut height = ((crop.height_percent as f64 / 100.0) * h_f).round() as u32;
+
+ if x >= img_w || y >= img_h {
+ return None;
+ }
+ width = width.clamp(1, img_w.saturating_sub(x).max(1));
+ height = height.clamp(1, img_h.saturating_sub(y).max(1));
+ Some((x, y, width, height))
+}
+
+fn maybe_crop_image(raw: &[u8], crop_region: Option<&BoundsPercent>) -> Option> {
+ let img = image::load_from_memory(raw).ok()?;
+ let (x, y, width, height) = crop_rect_px(img.width(), img.height(), crop_region)?;
+ let cropped = img.crop_imm(x, y, width, height);
+ let mut out = std::io::Cursor::new(Vec::new());
+ if cropped.write_to(&mut out, image::ImageFormat::Png).is_err() {
+ return None;
+ }
+ Some(out.into_inner())
+}
+
+/// Map click marker into cropped image coordinate space.
+/// Returns `None` when marker is outside the crop.
+pub fn marker_position_percent(step: &Step) -> Option<(f32, f32)> {
+ if step.screenshot_path.is_none() || is_auth_placeholder(step) {
+ return None;
+ }
+ let click_x = step.click_x_percent.clamp(0.0, 100.0);
+ let click_y = step.click_y_percent.clamp(0.0, 100.0);
+ let Some(crop) = normalize_crop_region(step.crop_region.as_ref()) else {
+ return Some((click_x, click_y));
+ };
+
+ let x = ((click_x - crop.x_percent) / crop.width_percent) * 100.0;
+ let y = ((click_y - crop.y_percent) / crop.height_percent) * 100.0;
+ if !(0.0..=100.0).contains(&x) || !(0.0..=100.0).contains(&y) {
+ return None;
+ }
+ Some((x.clamp(0.0, 100.0), y.clamp(0.0, 100.0)))
+}
+
/// Human-readable description of what happened in a step
pub fn action_description(step: &Step) -> String {
if is_auth_placeholder(step) {
- return "Authentication required (secure dialog)".to_string();
+ return "Authenticate with Touch ID or enter your password to continue.".to_string();
}
match step.action {
@@ -28,6 +122,15 @@ pub fn action_description(step: &Step) -> String {
}
}
+/// Prefer enhanced description if present, otherwise fall back to `action_description`.
+pub fn effective_description(step: &Step) -> String {
+ let desc = step.description.as_deref().unwrap_or("").trim();
+ if !desc.is_empty() {
+ return desc.to_string();
+ }
+ action_description(step)
+}
+
/// Image data with format metadata for export.
pub struct OptimizedImage {
pub bytes: Vec,
@@ -67,13 +170,29 @@ pub enum ImageTarget {
Pdf,
}
-/// Load a screenshot, convert to optimized format, return base64 + MIME.
-pub fn load_screenshot_optimized(path: &str, target: ImageTarget) -> Option<(String, &'static str)> {
+/// Load a screenshot and return optimized bytes + MIME/ext.
+pub fn load_screenshot_optimized_image(
+ path: &str,
+ target: ImageTarget,
+ crop_region: Option<&BoundsPercent>,
+) -> Option {
let raw = fs::read(path).ok()?;
+ let cropped = maybe_crop_image(&raw, crop_region);
+ let source = cropped.as_deref().unwrap_or(&raw);
let img = match target {
- ImageTarget::Web => to_webp_or_png(&raw),
- ImageTarget::Pdf => to_jpeg(&raw),
+ ImageTarget::Web => to_webp_or_png(source),
+ ImageTarget::Pdf => to_jpeg(source),
};
+ Some(img)
+}
+
+/// Load a screenshot, convert to optimized format, return base64 + MIME.
+pub fn load_screenshot_optimized(
+ path: &str,
+ target: ImageTarget,
+ crop_region: Option<&BoundsPercent>,
+) -> Option<(String, &'static str)> {
+ let img = load_screenshot_optimized_image(path, target, crop_region)?;
Some((
base64::engine::general_purpose::STANDARD.encode(&img.bytes),
img.mime,
@@ -88,12 +207,15 @@ pub fn to_jpeg(png_bytes: &[u8]) -> OptimizedImage {
let rgb = img.to_rgb8();
let mut buf = std::io::Cursor::new(Vec::new());
let encoder = image::codecs::jpeg::JpegEncoder::new_with_quality(&mut buf, 85);
- if encoder.write_image(
- rgb.as_raw(),
- rgb.width(),
- rgb.height(),
- image::ExtendedColorType::Rgb8,
- ).is_ok() {
+ if encoder
+ .write_image(
+ rgb.as_raw(),
+ rgb.width(),
+ rgb.height(),
+ image::ExtendedColorType::Rgb8,
+ )
+ .is_ok()
+ {
return OptimizedImage {
bytes: buf.into_inner(),
mime: "image/jpeg",
@@ -140,36 +262,54 @@ mod tests {
window_title: "Downloads".into(),
screenshot_path: None,
note: None,
+ description: None,
+ description_source: None,
+ description_status: None,
+ description_error: None,
+ ax: None,
capture_status: None,
capture_error: None,
+ crop_region: None,
}
}
#[test]
fn action_description_click() {
let s = sample_step();
- assert_eq!(action_description(&s), "Clicked in Finder \u{2014} \"Downloads\"");
+ assert_eq!(
+ action_description(&s),
+ "Clicked in Finder \u{2014} \"Downloads\""
+ );
}
#[test]
fn action_description_double_click() {
let mut s = sample_step();
s.action = ActionType::DoubleClick;
- assert_eq!(action_description(&s), "Double-clicked in Finder \u{2014} \"Downloads\"");
+ assert_eq!(
+ action_description(&s),
+ "Double-clicked in Finder \u{2014} \"Downloads\""
+ );
}
#[test]
fn action_description_right_click() {
let mut s = sample_step();
s.action = ActionType::RightClick;
- assert_eq!(action_description(&s), "Right-clicked in Finder \u{2014} \"Downloads\"");
+ assert_eq!(
+ action_description(&s),
+ "Right-clicked in Finder \u{2014} \"Downloads\""
+ );
}
#[test]
fn action_description_shortcut() {
let mut s = sample_step();
s.action = ActionType::Shortcut;
- assert_eq!(action_description(&s), "Used keyboard shortcut in Finder \u{2014} \"Downloads\"");
+ assert_eq!(
+ action_description(&s),
+ "Used keyboard shortcut in Finder \u{2014} \"Downloads\""
+ );
}
#[test]
@@ -184,14 +324,34 @@ mod tests {
fn action_description_auth_placeholder() {
let mut s = sample_step();
s.window_title = "Authentication dialog (secure)".into();
- assert_eq!(action_description(&s), "Authentication required (secure dialog)");
+ assert_eq!(
+ action_description(&s),
+ "Authenticate with Touch ID or enter your password to continue."
+ );
}
#[test]
fn action_description_auth_by_app() {
let mut s = sample_step();
s.app = "Authentication".into();
- assert_eq!(action_description(&s), "Authentication required (secure dialog)");
+ assert_eq!(
+ action_description(&s),
+ "Authenticate with Touch ID or enter your password to continue."
+ );
+ }
+
+ #[test]
+ fn effective_description_prefers_step_description() {
+ let mut s = sample_step();
+ s.description = Some("Click \"Share\"".into());
+ assert_eq!(effective_description(&s), "Click \"Share\"");
+ }
+
+ #[test]
+ fn effective_description_falls_back_when_empty() {
+ let mut s = sample_step();
+ s.description = Some(" ".into());
+ assert!(effective_description(&s).contains("Clicked in Finder"));
}
#[test]
@@ -214,7 +374,9 @@ mod tests {
let result = slugify_title("\u{00c4}rger mit \u{00d6}lf\u{00f6}rderung");
assert!(!result.contains('\u{00e4}'));
assert!(!result.contains('\u{00f6}'));
- assert!(result.chars().all(|c| c.is_ascii_alphanumeric() || c == '-'));
+ assert!(result
+ .chars()
+ .all(|c| c.is_ascii_alphanumeric() || c == '-'));
}
#[test]
@@ -261,7 +423,59 @@ mod tests {
#[test]
fn load_screenshot_optimized_missing_file() {
- assert!(load_screenshot_optimized("/nonexistent/file.png", ImageTarget::Web).is_none());
+ assert!(
+ load_screenshot_optimized("/nonexistent/file.png", ImageTarget::Web, None).is_none()
+ );
+ }
+
+ #[test]
+ fn marker_position_percent_without_crop() {
+ let mut s = sample_step();
+ s.screenshot_path = Some("/tmp/x.png".into());
+ s.click_x_percent = 70.0;
+ s.click_y_percent = 30.0;
+ assert_eq!(marker_position_percent(&s), Some((70.0, 30.0)));
+ }
+
+ #[test]
+ fn marker_position_percent_with_crop() {
+ let mut s = sample_step();
+ s.screenshot_path = Some("/tmp/x.png".into());
+ s.click_x_percent = 50.0;
+ s.click_y_percent = 50.0;
+ s.crop_region = Some(BoundsPercent {
+ x_percent: 25.0,
+ y_percent: 25.0,
+ width_percent: 50.0,
+ height_percent: 50.0,
+ });
+ assert_eq!(marker_position_percent(&s), Some((50.0, 50.0)));
+ }
+
+ #[test]
+ fn load_screenshot_optimized_image_applies_crop() {
+ use tempfile::TempDir;
+
+ let tmp = TempDir::new().unwrap();
+ let img = image::RgbaImage::from_pixel(200, 100, image::Rgba([255, 0, 0, 255]));
+ let img_path = tmp.path().join("shot.png");
+ img.save(&img_path).unwrap();
+
+ let out = load_screenshot_optimized_image(
+ img_path.to_str().unwrap(),
+ ImageTarget::Web,
+ Some(&BoundsPercent {
+ x_percent: 25.0,
+ y_percent: 20.0,
+ width_percent: 50.0,
+ height_percent: 50.0,
+ }),
+ )
+ .expect("optimized image");
+
+ let decoded = image::load_from_memory(&out.bytes).expect("decode optimized image");
+ assert_eq!(decoded.width(), 100);
+ assert_eq!(decoded.height(), 50);
}
#[test]
@@ -289,7 +503,7 @@ mod tests {
#[test]
fn jpeg_conversion_realistic_screenshot() {
- use image::{ImageFormat, RgbaImage, Rgba};
+ use image::{ImageFormat, Rgba, RgbaImage};
let (w, h) = (1440u32, 900u32);
let mut img = RgbaImage::new(w, h);
for y in 0..h {
@@ -308,18 +522,26 @@ mod tests {
let result = to_jpeg(&png_bytes);
let png_kb = png_bytes.len() / 1024;
let jpg_kb = result.bytes.len() / 1024;
- eprintln!("Screenshot 1440x900: PNG={png_kb}KB → JPEG={jpg_kb}KB ({}% savings)",
- if png_kb > 0 { 100 - (jpg_kb * 100 / png_kb) } else { 0 }
+ eprintln!(
+ "Screenshot 1440x900: PNG={png_kb}KB → JPEG={jpg_kb}KB ({}% savings)",
+ if png_kb > 0 {
+ 100 - (jpg_kb * 100 / png_kb)
+ } else {
+ 0
+ }
);
assert_eq!(result.mime, "image/jpeg");
- assert!(result.bytes.len() < png_bytes.len(), "JPEG should be smaller than PNG");
+ assert!(
+ result.bytes.len() < png_bytes.len(),
+ "JPEG should be smaller than PNG"
+ );
}
#[test]
fn webp_conversion_realistic_screenshot() {
// Simulate a 1440x900 screenshot with varied content (gradient + noise)
- use image::{ImageFormat, RgbaImage, Rgba};
+ use image::{ImageFormat, Rgba, RgbaImage};
let (w, h) = (1440u32, 900u32);
let mut img = RgbaImage::new(w, h);
for y in 0..h {
@@ -339,12 +561,20 @@ mod tests {
let png_kb = png_bytes.len() / 1024;
let out_kb = result.bytes.len() / 1024;
- eprintln!("Screenshot 1440x900: PNG={png_kb}KB → {}={out_kb}KB ({}% savings)",
+ eprintln!(
+ "Screenshot 1440x900: PNG={png_kb}KB → {}={out_kb}KB ({}% savings)",
result.ext,
- if png_kb > 0 { 100 - (out_kb * 100 / png_kb) } else { 0 }
+ if png_kb > 0 {
+ 100 - (out_kb * 100 / png_kb)
+ } else {
+ 0
+ }
);
- assert_eq!(result.ext, "webp", "WebP should be smaller for a realistic screenshot");
+ assert_eq!(
+ result.ext, "webp",
+ "WebP should be smaller for a realistic screenshot"
+ );
assert_eq!(result.mime, "image/webp");
// WebP should be meaningfully smaller (at least 20%)
assert!(
diff --git a/src-tauri/src/export/html.rs b/src-tauri/src/export/html.rs
index 130642c..8ca298f 100644
--- a/src-tauri/src/export/html.rs
+++ b/src-tauri/src/export/html.rs
@@ -1,5 +1,8 @@
+use super::helpers::{
+ effective_description, html_escape, load_screenshot_optimized, marker_position_percent,
+ ImageTarget,
+};
use crate::recorder::types::{ActionType, Step};
-use super::helpers::{action_description, html_escape, is_auth_placeholder, load_screenshot_optimized, ImageTarget};
/// Generate a self-contained HTML document from steps.
pub fn generate(title: &str, steps: &[Step]) -> String {
@@ -26,8 +29,13 @@ pub fn generate_for(title: &str, steps: &[Step], target: ImageTarget) -> String
-{title_esc} — {count} step{plural}
+
+
{title_esc}
+
{count} step{plural}
+
{steps_html}
+
+