frequent_items_sketch: add reset(), decide emptiness by preamble longs - #529
Merged
Merged
Conversation
- Add reset(), paralleling Java: returns the sketch to the empty state, keeping the maximum map size, equality operator and allocator. - On deserialize, emptiness is determined by preamble longs (1 or 4). The flags byte is only cross-checked against it; either legacy empty bit is still accepted. - Reject a non-empty image whose total weight is not positive. No valid writer produces it, and it would otherwise load as an empty sketch. - Document that no flag bits other than the empty bits are defined. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage Report for CI Build 35929114126Coverage increased (+0.05%) to 82.398%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
…age tests Longs and strings sketches with lg_max_map_size=8 and 193 distinct items: non-empty (N=193, offset=1) with no retained items, serialized with the full preamble. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tisonkun
approved these changes
Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #527, which made
is_empty()meantotal_weight == 0. A sketch whose purges cleared every counter is non-empty and now serializes with the full preamble and no items. This PR makes the read side and the API consistent with that.Changes
reset(), paralleling Java'sreset(). Returns the sketch to the empty state:total_weightandoffsetare zeroed and the map restarts atLG_MIN_MAP_SIZE. The maximum map size, equality operator and allocator are retained. Adds aget_equal()accessor toreverse_purge_hash_mapfor this.!(total_weight > 0)so NaN is rejected for floating-pointW.No change to the serialized format. Every image that deserialized before still deserializes, except the non-empty-with-zero-weight case above.
Tests
reset()after purges: empty, 8-byte image, max map size retained, and subsequent updates serialize identically to a newly constructed sketch.Cross-language images
The
[serialize_for_java]generator addsfrequent_long_purged_cpp.skandfrequent_string_purged_cpp.sk:lg_max_map_size=8and 193 distinct items, giving N=193, offset 1 and no retained items. Each is 32 bytes, preamble longs 4, flags 0. None of the existing images covers this form. apache/datasketches-java#770 reads these incheckCpp()and generates the Java counterparts; the C++ test that reads the Java images follows once that merges, sinceserde_compat.ymlgenerates from Javamain.All
fi_testcases pass, including[serde_compat]against the current Java-generated images.Related
Java counterpart: apache/datasketches-java#770 (
isEmpty()asstreamWeight == 0, the same read-side checks,FrequentLongsSketchstring format fixes, and the Java purged-to-zero images, which are byte-identical to the C++ ones).🤖 Generated with Claude Code