fix: 512px VAE decode OOM - effective half-size tiling + unconditional retry - #1932
Open
Hmission wants to merge 1 commit into
Open
fix: 512px VAE decode OOM - effective half-size tiling + unconditional retry#1932Hmission wants to merge 1 commit into
Hmission wants to merge 1 commit into
Conversation
…l retry - backend_fit: prepare_vae_decode_retry_tiling now defaults rel_size to 0.5 when enabling spatial tiling. get_tile_sizes() defaulted to rel_size=1.0 (factor branch wins) which produced a full-latent tile - tiling was a no-op and 512px decode could exceed device buffer limits (Adreno 740 ~1.94GB -> ~416MB with half tiles). - stable-diffusion: retry VAE decode with tiling on decode failure (empty result, typically OOM) regardless of --auto-fit, so small-buffer mobile GPUs recover automatically. Fires only on failure; happy path unchanged. Evidence (Pocket Chick, K Pad Mali-G925 / Adreno 740): - 512px decode: 1.94GB -> 416MB buffer, renders green channel correctly (prior NaN/white-image on SD3.5 OpenCL); 10-step ~45.8s (0.4% accuracy loss vs non-tiled on K90), 13 tiles; Z-Image K90 39.7s.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two small fixes so 512px VAE decode fits small mobile GPU buffers without resorting to full-image fallbacks:
effective tiling (
src/core/backend_fit.cpp):prepare_vae_decode_retry_tiling()enables spatial tiling but did not setrel_size_x/y, andget_tile_sizes()defaults torel_size=1.0when the factor branch wins - so the "tile" was the full latent (tiling a no-op). Defaultrel_sizeto 0.5 when enabling spatial tiling: 512px decode goes from ~1.94GB to ~416MB of buffers on Adreno 740, with standard VAE tiling overlap (no quality impact, measured 0.4% accuracy delta vs non-tiled on K90 10-step).unconditional retry (
src/stable-diffusion.cpp): retry VAE decode with tiling whendecoded.empty()(typically OOM) regardless of--auto-fit, which defaults to off. Without this, small-buffer devices just get a black/empty image unless the user happens to pass--auto-fit. The retry only fires after a failed decode - the happy path is unchanged, andprepare_vae_decode_retry_tilingstill returns false on second failure so the loop terminates.Motivation
Pocket Chick (React Native app, forked from PocketPal) runs SD3.5 / Z-Image-Turbo on Android phones (Mali-G925, Adreno 740). At 512px the OpenCL decode graph previously exceeded device buffer limits (1.94GB requested) and produced blank/NaN output on some devices. These two fixes are the minimal upstream-friendly version of what we validated on device.
Evidence (on-device, K Pad Mali-G925 / Adreno 740)
Scope
--auto-fitbehavior intentionally broadened (see summary 2).