Repository navigation
perf(session): prefilter and parallelize session search - #25
Merged
Merged
Conversation
Search parsed every candidate transcript sequentially before matching. Now it: - skips transcripts whose raw bytes lack any ASCII word of a literal query (regex queries are not prefiltered) - scans candidates in parallel chunks with rayon, keeping newest-first order and early stop on --limit/--scan - loads sessions from the path recorded at listing time instead of re-walking storage by ID Full scans over 535 sessions drop from ~2.5s to 0.35-0.9s. Loading by path also fixes Gemini sessions that share an ID returning the same file repeatedly. The JSON report gains a `prefiltered` count.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
agentspec session searchparsed every candidate transcript one at a time before matching. This change makes full scans 3-8x faster.--limit/--scanstill stop early. Prefiltered sessions do not count toward--scan.SessionMetaIRrecordssource_path(not serialized), and a newSessionAdapter::load_from_metaparses it directly instead of re-walking storage by ID.prefilteredcount.Benchmark (535 sessions, warm cache, release build)
cargo"rate limit" --role user--regex 'cargo\s+build'--tool-used BashResult sets matched the old binary on all six benchmark queries, except one bug fix: three Gemini files sharing session ID
4e24b246…were previously all loaded from the first file found. They now load from their own files.Test plan
cargo fmt --check,cargo clippy --all-targets -D warningscargo test(158 passing, 2 new prefilter tests)