Repository navigation
fix: hash transparent images as they look on a white background - #92
Merged
Merged
Conversation
Grayscale conversion ignores alpha, so transparent pixels were hashed by whatever colour was stored behind them, and palette PNGs with byte transparency made Pillow emit a UserWarning on every scan. Co-Authored-By: Toldry <toldry@gmail.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CAPbY36A2C1ujRpj8CZWgi
Co-Authored-By: Toldry <toldry@gmail.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EGsDrajpL6puoVuiCcZvoR
Co-Authored-By: Toldry <toldry@gmail.com> Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EGsDrajpL6puoVuiCcZvoR
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Querent · project thread
Before: when the image duplicate scan met a palette PNG with partial transparency, Pillow logged
UserWarning: Palette images with Transparency expressed in bytes should be converted to RGBA imagesin production. More importantly, every transparent image was hashed by the hidden colour stored behind its see-through pixels (grayscale conversion ignores alpha), so two visually identical copies saved by different programs could get completely different hashes and not match.After: transparent images are laid over a white background before hashing, so the scan compares what a viewer sees, and the warning no longer appears. Opaque images are hashed exactly as before.
How: a new
flatten_transparency()induplicate_detector.pyalpha-composites any image withhas_transparency_dataonto white (via RGBA, the conversion Pillow's warning asks for) before both the whole-image and region dhashes. Hashes are only cached in memory, not stored in the database, so nothing needs migrating. Two unit tests: a transparent palette PNG with a black hidden background, one with a white hidden background, and a flattened JPEG all match, with warnings turned into errors; and opaque images pass through untouched. The first test fails onmaster. Full unit suite: 1699 passed.🤖 Generated with Claude Code
https://claude.ai/code/session_01CAPbY36A2C1ujRpj8CZWgi
Generated by Claude Code