Skip to content

feat(processor): index Rapid Log entries and sync handwritten tasks into ScheduleService (#162) - #473

Open
allenporter wants to merge 1 commit into
mainfrom
feat/magic-todo-sync-162
Open

allenporter wants to merge 1 commit into
mainfrom
feat/magic-todo-sync-162

Conversation

@allenporter

@allenporter allenporter commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Fixes #162

Summary

Builds on the Bullet Journal Rapid Logging parser (#472) to index structured Rapid Logging entries (TASK, EVENT, NOTE) from OCR'd .note pages into SQLite (f_rapid_log_entry) and automatically reconcile handwritten tasks with ScheduleService (including page deep-links and cross-page migration).

Key Changes

  • Structured Rapid Log Index (RapidLogEntryDO / f_rapid_log_entry):
    • Persists all parsed Rapid Logging items (kind: TASK, EVENT (o), NOTE (-)) with text, normalized_text, detail, task_state, is_starred, is_cancelled, page_date_ms (via infer_page_date(page.page_id)), page_index, content_hash, and schedule_task_id (Alembic revision e6f7a8b9c0d1).
    • Avoids naming collisions with background SystemTaskDO tasks while making dated EVENT (o) items immediately queryable in SQLite for future dynamic .ics calendar export and MCP tools without another schema migration or re-OCR pass.
  • content_hash Change Detection & ScheduleService Sync (RapidLogModule):
    • Registered in ProcessorService.global_post_modules (TaskType.RAPID_LOG_EXTRACTION).
    • Uses per-entry content_hash to skip unchanged items in O(1) without querying ScheduleService, preserving digital completions and digital deletions automatically without tombstone state machines.
    • Creates open tasks (•, >, <, [ ]) in a lazily created "From Notes" ScheduleTaskGroupDO with Base64 ScheduleTaskLinkDTO deep-links (appName="Note", 1-based page, fileId, pageId, taskTitle), importance="high" for * signifiers, nested detail, and due_time from page_date_ms (ignoring pre-completed X and pre-cancelled ~~...~~ items on first sight).
    • Transfers schedule_task_id ownership across pages on Bullet Journal migrations (> / < -> • or same-title open tasks) to advance deep-links and due_time without duplicating tasks.
    • Cleans up open synced ScheduleTaskDOs and RapidLogEntryDO rows on strikethrough (~~...~~), lasso erasure, and NoteDeletedEvent.
  • Generic Global Checkpoint Invalidation (PageHashingModule):
    • Invalidates all key="global" SystemTaskDO rows (task_type != PAGE_HASHING) whenever any page changes, without coupling PageHashingModule to downstream module task types.

Verification

  • ./script/test across test_rapid_log.py (utils & processor module), test_page_hashing.py, test_processor.py, test_migrations.py, test_init.py
  • ./script/lint (ruff, ty, codespell, yamllint)

@codecov-commenter

codecov-commenter commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.55224% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.40%. Comparing base (570faee) to head (56a06e3).

Files with missing lines Patch % Lines
...ote/server/services/processor_modules/rapid_log.py 86.68% 55 Missing ⚠️
supernote/server/services/processor.py 98.07% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #473      +/-   ##
==========================================
- Coverage   94.57%   94.40%   -0.18%     
==========================================
  Files         155      157       +2     
  Lines       18667    19168     +501     
==========================================
+ Hits        17655    18096     +441     
- Misses       1012     1072      +60     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@allenporter allenporter added enhancement User-facing feature or API capability ai OCR, summarization, embeddings, prompts, vocabulary, and AI backends schedule Schedule, To-Do tasks, calendar feeds (.ics), and CalDAV labels Nov 1, 2026
@allenporter
allenporter force-pushed the feat/magic-todo-sync-162 branch from da0e16e to 7b4db1c Compare November 8, 2026 05:55
@allenporter allenporter changed the title feat(processor): extract handwritten tasks into ScheduleService with page deep-links (#162) feat(processor): extract handwritten to-dos into ScheduleService with page deep-links (#162) Nov 8, 2026
@allenporter
allenporter force-pushed the feat/magic-todo-sync-162 branch 2 times, most recently from da875e9 to 32c96b5 Compare November 8, 2026 06:31
@allenporter allenporter changed the title feat(processor): extract handwritten to-dos into ScheduleService with page deep-links (#162) feat(processor): index Rapid Log entries and sync handwritten tasks into ScheduleService (#162) Nov 8, 2026
@allenporter
allenporter force-pushed the feat/magic-todo-sync-162 branch from 32c96b5 to d20e49f Compare November 8, 2026 06:46
@allenporter
allenporter force-pushed the feat/magic-todo-sync-162 branch from d20e49f to 56a06e3 Compare November 8, 2026 07:02

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai OCR, summarization, embeddings, prompts, vocabulary, and AI backends enhancement User-facing feature or API capability schedule Schedule, To-Do tasks, calendar feeds (.ics), and CalDAV

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(processor): extract handwritten action items and checkboxes into ScheduleService with notebook page deep-links

2 participants